Anonymous
Ah .. let me check the link
Alexander
If you okay with redeclare entities, you can specify table name for each entity by youself
Anonymous
If you okay with redeclare entities, you can specify table name for each entity by youself
I need a tool to fast converting
Alexander
So take a look at pony-inspect
Anonymous
Yes I will. Is it being tested and ready to be used ?
Alexander
It should work for simple cases, but entity declarations sometimes may be not ideal, for example instead of many-to-many relationship you will get intermediate entity. So you may need clean up entities manually at the end
Anonymous
Yes I had read it that python-inspect has limitations so I probably should specify tables into pony class manually
Anonymous
Is there a way to import existing pony database into pony ERD?
Alexander
At this moment, no
Anonymous
Okay
Grigory
Hi guys!
` File "<auto generated wrapper of process_channel_dir_file() function>", line 2, in process_channel_dir_file
File "/home/vader/.local/lib/python2.7/site-packages/pony/orm/core.py", line 480, in new_func
reraise(exc_type, exc, tb)
File "/home/vader/.local/lib/python2.7/site-packages/pony/orm/core.py", line 468, in new_func
commit()
File "/home/vader/.local/lib/python2.7/site-packages/pony/orm/core.py", line 349, in commit
transact_reraise(CommitException, exceptions)
File "/home/vader/.local/lib/python2.7/site-packages/pony/orm/core.py", line 319, in transact_reraise
reraise(exc_class, new_exc, tb)
File "/home/vader/.local/lib/python2.7/site-packages/pony/orm/core.py", line 343, in commit
primary_cache.commit()
File "/home/vader/.local/lib/python2.7/site-packages/pony/orm/core.py", line 1727, in commit
cache.database.provider.commit(cache.connection, cache)
File "/home/vader/.local/lib/python2.7/site-packages/pony/orm/dbproviders/sqlite.py", line 315, in commit
provider.transaction_lock.release()
CommitException: error: release unlocked lock`
This error appears when I return a new ORM object creater @db_session decorated function to another @db_session decorated function.
Anyone knows how to handle this?
Grigory
Jorge
you make a dbsession in the scope of another dbsession?
Grigory
no. I return object from one method to another.
Grigory
Both methods a wrapped in db_session
Grigory
And this happens only if I call some_binding.exists(foo=bar) before return
Grigory
i.e. something inside .exists releases the lock
Grigory
and it happens on .get too
Grigory
but does not happen on .select
Henri
Do you have an example?
Grigory
it's a long one, in our codebase. I'll try to make a minimal now.
Alexander
Hi Grigori! Do you use async libraries like Tornado in your code?
Grigory
Twisted, that is
Grigory
btw, I can't reproduce it cleanly from bottom up
Grigory
but if I run any select before exists, the bug goes away
Grigory
and it is 100% reproducible on our code base
Grigory
nothing asyncrhonous is used in this case. It's just unit test directly calling some method.
Grigory
both the unit test and the method are wrapped in db_session
Alexander
So, no async functions are decorated with db_session?
Grigory
no
Alexander
I need some time to understand what's going on
Grigory
I kinda minimized it
Grigory
the outer function:
` @db_session
def test_bug(self):
t = self.metadata_store.TorrentMetadata(title='test')
t.delete()
self.metadata_store.bug(
`
Inner function:
` @db_session
def bug(self):
self.Metadata.exists(signature="123")
return self.TorrentMetadata(title='dfdfd')
`
Grigory
however, I still can't reproduce it cleanly without our stack
Grigory
this delete is required to trigger the bug
Grigory
if I insert select(...)[:] before exists, bug goes away
Grigory
yay!
Grigory
I got a minimal example!
Grigory
`from pony import orm
from pony.orm import db_session
db = orm.Database()
class TestClass1(db.Entity):
rowid = orm.PrimaryKey(int, auto=True)
val = orm.Optional(int, default=0)
db.bind(provider='sqlite', filename=":memory:", create_db=True)
db.generate_mapping(create_tables=True)
@db_session
def fun1():
TestClass1.exists(val=12)
return TestClass1()
@db_session
def fun2():
a = TestClass1()
a.delete()
q = fun1()
print q
def main():
with db_session:
fun2()
if name == "main":
main()
`
Grigory
i'll post it on issue tracker
Alexander
ok, cool
Grigory
https://github.com/ponyorm/pony/issues/386
Grigory
here you are, sir!
Alexander
Thanks for reporting, I hope I will be able to fix it today
Grigory
It was really nice when you fixed the bug I reported the last time!
Thanks!
Grigory
Grigory
@metaprogrammer wow, that was a quick job of that bug!
Anonymous
It's already fixed? Wow!
Alexander
yes
Anonymous
I now feel excused then to ask another of my nitpicking questions: is it possible to use startswith on a list of options somehow, or would it need patching it in Pony?
The following doesn't work:
subclasses = tuple(c.__name__ for c in A.__subclasses__())
select(l for l in Log if l.representation.startswith(subclasses))According to Python's documentation, startswith accepts tuples, but it doesn't work in Pony.
I tried an alternative way, but it doesn't work either:
subclasses = tuple(c.__name__ for c in A.__subclasses__())
select(l for l in Log if any(l.representation.startswith(c) for c in subclasses))This one has a pretty obvious reason, as I try to iterate over c, which is not a database entity, inside a query.
Anonymous
By the way, you may notice some similarities to the matter I raised here: the isinstance function. But here, in this case, I really need to use repr and eval, because I try to make a Log object, which accepts any valid db.Entity as one of its values.
So I'll ask two more questions about this:
1. Can I count on repr to always return strings that are reversible to entity in eval?
2. Would it be possible to add the option of having Set(db.Entity) (or Required(db.Entity) )?
You may also remember the issue about primary key I mentioned here; since it's not yet fixed, I need to use repr, which works correctly.
Anonymous
# If you prefer, I can move that to GitHub as a question, so it would stay there for future reference for others.
Alexander
Hi Krzysztof! Why do you need to use startswith instead of equality check, like log.classname in subclasses?
Alexander
In principle it is possible to add support of startswith with a list of prefixes, but the resulted SQL will not be efficient
Anonymous
I didn't think of that, I suppose I could add classname field to Log instead of just representation. However, if I would need to store multiple representations, I could always separate them by semicolons or something, but in this case I would need to create a separate Entity class with classname and representation as attributes, and add Set(Entity) to Log. I have a feeling that would be inefficient, but that's just a hunch – what do you think?
Anonymous
The best solution, I think, would be to add Set(db.Entity) possibility, which could work with strings and representation under the hood, just like I'm thinking of doing right now.
Anonymous
Or maybe you have something even better in mind?
Alexander
> Can I count on repr to always return strings that are reversible to entity in eval?
I think yes, if you don't redeclare __repr__ in specific entity class. But it looks like a hack
> Would it be possible to add the option of having Set(db.Entity) or Required(db.Entity)?
At this moment generic relationships are not supported. It is possible to add them, but it is a non-trivial task.
If all your entity classes use simple id value as a primary key, and each log record linked to a single object of arbitrary type, then you can just define two attributes, like obj_class and obj_id, and then get object using db.entities[log.obj_class][obj_id]
Alexander
It is not entirely clear what goal your logging system should achieve, so it is hard to give specific advice. Maybe you can use the following:
class LogRecord(db.Entity):
id = PrimaryKey(int, auto=True)
dt = Required(datetime)
description = Required(str)
objects = Set("ObjectRef")
class ObjectRef(db.Entity):
obj_class = Required(str)
obj_id = Required(int)
log_records = Set("LogRecord")
PrimaryKey(obj_class, obj_id)
And then you create an instance of AnyObject for each object of another type
p1 = Person(name='John', age=22)
p1_ref = ObjectRef(obj_class=p1.__class__.__name__, obj_id=p1.id)
log_record = LogRecord(description="Person created", objects=[p1_ref])
And then you can find ObjectRef instances related to specific log records, and found real objects corresponding to ObjectRef instances
Anonymous
Well, not all my objects use plain id for primary keys, so I can't use that, I would really need to stick to repr, because then I can have things like A[B[1],C[2]]. So I think I'll have to use another (meta-)entity for that, which would look pretty much like what you did above.
Alexander
Then maybe
class ObjectRef(db.Entity):
repr = PrimaryKey(str)
classname = Required(str)
log_records = Set("LogRecord")
Anonymous
Yeah, that's the way I'm going to do that, I suppose. So I think that matter is resolved, I should really have thought that through before asking. Lesson learned.
Anonymous
But, since we're already here, what do you think about adding something like OrderedSet as a feature? Right now, if we want to have a set of objects with their order, we have to do it like this:
class Track(db.Entity):
id = PrimaryKey(int, auto = True)
title = Required(str)
artist = Required('Artist')
playlist_appearances = Set('TrackAppearance')
class TrackAppearance(db.Entity):
playlist = Required('Playlist')
track = Required(Track)
position = Required(int)
PrimaryKey(playlist, track)
class Playlist(db.Entity):
id = PrimaryKey(int, auto = True)
name = Required(str)
tracks = Set(TrackAppearance)Is it really that inefficient as it looks, or maybe it's not that bad? I have several relationships like that in my database, and I don't think I can really change them.
Alexander
It is a good idea, and I wanted to add a List attribute type from the beginning, which internally uses an intermediate table like in your example. Maybe OrderedSet is a better name for it. I definitely want to add it, but it is a big task, so probably not in the near future
Alexander
Right now you need to implement OrderedSet manually using intermediate entity like in your example
Alexander
Note that when using intermediate entity like TrackAppearance you can use attribute lifting to get all Playlists related to specific Track if order is not important: track.playlist_appearances.playlist gives a set of Playlist objects. And in reverse direction, playlist.tracks.track returns a set of all Track objects related to specific playlist
Alexander
It simplifies some queries. But manualy working with position value is still tedious, so I hope OrderedSet will be implemented eventualy
Anonymous
Alright, thank you very much! I feel more confident in that solution now. :-) Looking forward for the OrderedSet or List in the future, but no rush, it's not that big deal. :-)
stsouko
I would like to offer a new feature. raw data types. operators to which can be choosen from the available list
stsouko
This give a possibility to add db specific types
Anonymous
Could you give an example of such type?
stsouko
Pistgre arrays
Anonymous
But that would break the independence of Pony-generated scheme with other databases, wouldn't it?
Anonymous
I'm no expert, I'm just curious.
Anonymous
By the way, using raw SQL queries wouldn't help you?
stsouko
I want to transparently store and load from db sets of digits without json
stsouko
Arrays give subsets and simularity search
Anonymous
I have a feeling that it won't be suitable for Pony as it is an engine-dependent feature. But you'll have to wait for @metaprogrammer to answer this.
Anonymous
By the way, I believe I found a very minor bug – or at least an incorrect example in the documentation.