Lucky
Anonymous
@here any idea why pony would ever throw an
AttributeError: pony.orm.dbproviders.sqlite doesn't have attribute provider_cls
?
Lucky
Hey, I'm havin unexpected behaviour with bool:
packs = orm.select(
p for p in Pack if not p.nsfw
)
Results in
SELECT "p"."url", "p"."title", "p"."nsfw", "p"."owner_id", "p"."first_seen", "p"."last_crawled", "p"."sticker_count"
FROM "pack" "p"
WHERE ("p"."nsfw")::int = 0
Which doesn't seem to cover the not None=>True case.
Alexander
Thanks for reporting. That's a bug, I'll fix it
Alexander
Lucky
Not sure what postgres does with null::int
Alexander
Hey team! How's migration going?
Was looking to change a 0,1---0,1 relationship to 0,n---0,1
The operation "change type of relation" have many complex cases and will not be fully covered in first version of migration tool. So you will have to do that change from "one-to-one" to "one-to-many" manually. In order to do this, you need to perform the following command:
alter table websession add column user int;
After you start application the index and foreign key will be added automatically if db.generate_mapping method is called with create_tables=True option.
You alse need to perform that update query:
update websession w set user = (select id from "user" where web_session = w.id)
Alexander
Lucky
Indeed, workaround is:
p for p in Pack if p.nsfw is True or p.nsfw is None
Micaiah
Is there a better way to ignore an argument from a Table.get call than using if statements
Micaiah
something like:
Table.get(name=name, version=version, ignore_on_none=True)
Micaiah
so if name or version is None it'll just pretend it wasn't passed
Micaiah
(if i'm just being dumb its okay to say that instead)
Alexander
You can write something like that:
def ignore_none(**kwargs):
return {key: value for key, value in kwargs.items()
if value is not None}
Table.get(**ignore_none(name=name, version=version))
Micaiah
Micaiah
I'm getting a TypeError from sqltranslations. "ignore_none_select cannot be used this way"
Micaiah
Micaiah
The function itself seems to work, I tested it outside of a pony transaction and got the expected results
Alexander
Well... you asked about get, and select is entirely different story. Pony should translate select generator code to SQL, and cannot translate your function, because it doesn't know what equivalent SQL should be. I don't know it too :)
Also, the function that you wrote expects keyword arguments, not positional ones as you use inside the query.
1) I suggest you to write selects explicitly:
games = select(g for g in Game if (g.name == name or name is None) and (g.version == version or version is None))
2) You can also build query incrementally:
query = select(g for g in Game)
if name is not None:
query = query.filter(lambda g: g.name == name)
if version is not None:
query = query.filter(lambda g: g.version == version)
3) Probably even something like that should work:
def ignore_none(query, **kwargs):
for key, value in kwargs.items():
if value is not None:
query = query.filter(lambda x: getattr(x, key) == value)
return query
query = select(g for g in Game)
query = ignore_none(query, name='X', version='Y')
I didn't test it. Theoretically it should work, but there is an open issue 223, which describes a bug with getattr which is not fixed yet. You may follow approach (2) for safety
Lucky
Well... you asked about get, and select is entirely different story. Pony should translate select generator code to SQL, and cannot translate your function, because it doesn't know what equivalent SQL should be. I don't know it too :)
Also, the function that you wrote expects keyword arguments, not positional ones as you use inside the query.
1) I suggest you to write selects explicitly:
games = select(g for g in Game if (g.name == name or name is None) and (g.version == version or version is None))
2) You can also build query incrementally:
query = select(g for g in Game)
if name is not None:
query = query.filter(lambda g: g.name == name)
if version is not None:
query = query.filter(lambda g: g.version == version)
3) Probably even something like that should work:
def ignore_none(query, **kwargs):
for key, value in kwargs.items():
if value is not None:
query = query.filter(lambda x: getattr(x, key) == value)
return query
query = select(g for g in Game)
query = ignore_none(query, name='X', version='Y')
I didn't test it. Theoretically it should work, but there is an open issue 223, which describes a bug with getattr which is not fixed yet. You may follow approach (2) for safety
How would I execute (2) ?
Alexander
name = 'X'
version = 'Y'
query = select(g for g in Game)
if name is not None:
query = query.filter(lambda g: g.name == name)
if version is not None:
query = query.filter(lambda g: g.version == version)
objects = query[:]
Lucky
Could you guy please start with code documentation?
Most importantly this would be for the public methods, so IDEs like Pycharm could give quick documentation.
Also reading the source is really hard this way.
Lucky
Lucky
I mean, you could just add a description whenever you edit somethin in that function anyway.
Lucky
Also stuff like .to_json() is not documented online either:
https://docs.ponyorm.com/api_reference.html?highlight=to_json#Query.to_json
Святослав
I can't specify order for columns in composit_index?
Святослав
it's not important, i just ask you plan or not adding tools for customization models
Святослав
I remember you make a migration tool
Alexander
If you specify composite_index(attr1, attr2), the columns will be in that order
Святослав
Writing raw sql and used for deploy will be great. And if i right remember, it will be released with migration tool?
Святослав
Alexander
Yes, you will be able to write raw sql in migrations
Alexander
@luckydonald so you mean add docstrings to all functions?
Lucky
Lucky
Lucky
Will `_get_schema_dict` (`to_json()["mapping"]`) and `.to_dict` be removed too?
Lucky
Alexander
The problem with to_json is that it has weak possibilities in querying hierarchical data. We want to replace it with something like that:
select(g for g in UniversityGroup if g.graduate_year == 2018).extract("""
number,
major,
department {
number,
name,
faculty { name }
},
courses foreach(c) orderby(c.name) {
id, name, credits
}
students foreach(s) where(s.gpa > 3.5) orderby(s.name) limit(10) {
id, name, gpa
},
""")
Lucky
The problem with to_json is that it has weak possibilities in querying hierarchical data. We want to replace it with something like that:
select(g for g in UniversityGroup if g.graduate_year == 2018).extract("""
number,
major,
department {
number,
name,
faculty { name }
},
courses foreach(c) orderby(c.name) {
id, name, credits
}
students foreach(s) where(s.gpa > 3.5) orderby(s.name) limit(10) {
id, name, gpa
},
""")
With that you mean, that there are only the ids of other tables? And the plan is to actually include the subdata there?
Actually I like how it works with reference keys now.
Lucky
The problem with to_json is that it has weak possibilities in querying hierarchical data. We want to replace it with something like that:
select(g for g in UniversityGroup if g.graduate_year == 2018).extract("""
number,
major,
department {
number,
name,
faculty { name }
},
courses foreach(c) orderby(c.name) {
id, name, credits
}
students foreach(s) where(s.gpa > 3.5) orderby(s.name) limit(10) {
id, name, gpa
},
""")
Also this looks neither like python nor like sql
I'm not sure if inventing your own query language for that is a good idea.
Alexander
This based on our experience we got during developing the site fineartbiblio.com.
Sometimes it is useful to load complex hierarhical data following object relationships. For example, on art gallery site page you want to load 10 most popular artists, and for each of them load some specific fields, like name and year of birth, and also load several related collections - 10 most significant artworks, 5 most recent catalogs, and for each catalog load 3 illustrations, and for each artwork load list of provenance records, etc. As a result you will get some hierarchical JSON structure.
This requires some hierarchical query API. There are several ways how it may be done.
One way is to use some pure-Python constructions. We are experimented with this, and the result was disappointing. Either the API looks very ugly, or its insufficiently powerful when working with subcollections.
Another way is to use some existing hierarchical query API, like GraphQL. It is possible, but often looks overcomplicate ("cursors" and all that stuff) and too heavyweight.
The third way is to use custom query language with easy to understand syntax, which allows to embed arbitrary Python expressions for conditions. After experimenting with other approaches, I believe this is the most convenient way.
Alexander
> Actually I like how it works with reference keys now.
So you are already use to_json? Can you tell a bit more how you use it? Do you manually process resulted data structure?
Actually, the data format of to_json result is pretty good. But it was intened to use with corresponding PonyJS library which uses that data to construct JavaScript obejcts which are equivalent to Python objects. Do you use unreleased version of PonyJS, or process data manually instead?
Alexander
What I dislike in to_json function is not the data format, but the lack of expressive power when it is necessary to recursively specify fields and subcollections which need to be retrieved
Lucky
We don't use it already, but @mshekhter and I am looking to build an adapter to put sql data to an elasticsearch index. And the data structure returned looks perfect already. All left to to would be to map the datatypes to an elastic equivalent.
From what we found so far, we'd could use the json of the elements unchanged to push the data into the ES index.
Lucky
Alexey
@luckydonald did you try https://docs.ponyorm.com/api_reference.html#Entity.to_dict
may be this will work for you?
Lucky
Can I specify that on a query restult too?
But probably yes.
Lucky
In conjunction with database._get_schema_dict() it is basicly the same.
Alexander
But why implement another lanuage if you could just write python?
We don't found expressive enough way to write hierarchical queries using pure Python, maybe you can suggest it. I think custom DSL with embedded Python fragments provides the good compromise between familiarity and expressivity.
But in the nearest perspective you can use to_dict(), maybe it fully suits you needs
Micaiah
anything with flask or hug that i do uses to_dict() extensively
Rozen
Hi, i'm having... like..
Rozen
and odd error
Rozen
i have this function
Rozen
@db_session
def join(user, group):
jugador = Jugador.get(user=user,group=group)
res = NOTHING
if (not jugador):
user = User.get(id_user=user.id_user)
user = select( u for u in User if u.id_user == user.id_user)[:][0]
print(user.id_user)
jugador = Jugador(user=user,group=group)
juego = Juego.get(group=group)
if (jugador not in juego.jugadores):
juego.jugadores.add(jugador)
res = JOINED
return res
Rozen
(sorry for not pastebin)
Rozen
user and group are preloaded objects from the table
Rozen
when i print the id_user i get the number
Rozen
but in this line jugador = Jugador(user=user,group=group)
Rozen
i get ValueError: Attribute User.id_user is required
Micaiah
Is the id_user set to PrimaryKey?
Rozen
Yep
Micaiah
Hmm, strange.
Alexander
Can you provide full traceback?
Rozen
In 5 mins
Alexander
I think this is what happened:
1) Jugador class derived from User
2) User class has id_user column
3) This column does not have auto=True option
4) You create Jugador instance without specifying id_user value
Alexander
You need to specify auto=True option when defining id_user attribute. Or you can omiss definition of id_user at all, and pony creates default id attribute with auto=True
Rozen
oh ..
Rozen
it was the 1 option
Rozen
i don't know what happened but i wanted jugador nor to be a child class from User
Rozen
well ... i think im a mess 😆
Rozen
Thanks!!
Alexander
Sure!
Pavel WTF
Hi all.
I am new to relational dbs and have some questions.
I have existing project with mongodb and my own data access objects. I want to migrate it to postgresql using ponyorm. I already created db model and done with data migration.
The question is:
What is the best way to integrate pony in existing code?
Should I replace existing models?
Alexander
I think it will be much easier to replace models than to use some wrappers
Artur Rakhmatulin
Hi!
If you use your customORM like CRUD and no more, I think you will have no problems if you replace your 'models'.
it would be better if you have tests