Alexander
Hello, Alexander! I want to ask your opinion as co-founder of Pony: what is the best approach to implement multi-language in DB architecture using Pony? Suppose I'm designing a diagram of some CMS, and I have a Document entity with following attributes: title, description, content (omit all unnecessary). All of them should be represented in all languages defined in the system (via Language entity). I see several approaches: 1) To use Json type for such columns, i.e. Document[1].title = {'ru': 'Тестовый документ', 'en': 'Test document'} Pros: - easy to retrieve data w/o joining additional tables Cons: - might not be suitable for content which can store too large data 2.1) To use additional entity like String which related to Language (1:M relation type) and to all of the fields of Document via M:1, i.e.: Document(title=[String(lang='en', value='Test document'), String(lang='ru', value='Тестовый документ')]) Pros: - seems it is most flexible way Cons: - the String entity should contain back-refs to each field of Document (title, description, content). But what if we need to localize another entities, like Template, Chunk, Property etc? I think String definition will be too messy. - more complex select with join, like select((d, t) for d in Document for t in Translation if t.language.code == 'en') - not quite convenient way to access localized strings as in (1) 2.2) To use intermediary entity Translation which encapsulates all of the fields to be localized. Maybe also to use inheritance for particular entities, like DocumentTranslation, PropertyTranslation etc which will add specific attributes if it needed. Pros: - organize messy code of abstract String entity Cons: - the same as in (2.1) 3) To use hardcoded attributes like title_ru, title_en, content_ru, content_en. Pros: - easy access to attributes w/o joining Cons: - not so easy to add new language Maybe I'm missing something? What can you advice? Thank you!
It is hard to suggest something specific, as it may depend on what is the most typical scenarios. Like, do you have some "primary" language or not. Maybe I'd use something like: class TranslationsMixin: DEFAULT_LANG = 'en' def setfield(self, attrname, lang, value): if lang == self.DEFAULT_LANG: setattr(self, attrname, value) else: if not self.info: self.translations = {} self.translations.setdefault(lang, {})[attrname] = value def getfield(self, attrname, lang): if lang = self.DEFAULT_LANG: return getattr(self, attrname) else: return self.translations.get(lang, {}).get(attrname) class Person(db.Entity, TranslationsMixin): first_name = Required(str) last_name = Required(str) age = Required(int) translations = Required(Json)
Anonymous
Hi everyone! Could anybody tell usecases in anyio/asyncio - I have a projects where async code is prefered ? And what about migrations ??? It would be nice to have a support of django-like migrations….
Alexander
Hi! PonyORM does not support async at this moment. For async applications I suggest you to use something like TortoiseORM instead https://github.com/tortoise/tortoise-orm Currently work on migrations are paused, I hope to resume it a bit later
Anonymous
Hi! PonyORM does not support async at this moment. For async applications I suggest you to use something like TortoiseORM instead https://github.com/tortoise/tortoise-orm Currently work on migrations are paused, I hope to resume it a bit later
Thank you for reply. But, for my customers - It's comfortably to show models in DB designer - preferrebly online - so PonyOrm has one and I would like to not use 2 different solutions - first for customers - second for coding - I hope there is any painless solution to use ponyorm in async code ....
Alexander
You need to be careful then: - only use db_session context manager, not db_session decorator with async code - never call async functions (await foo()) inside db_session
Anonymous
You need to be careful then: - only use db_session context manager, not db_session decorator with async code - never call async functions (await foo()) inside db_session
Thank you 😊 as I understood - db_session context manager is synchronized operation. Just thought, but if I write simple convertor from ponyorm form to tortoise-orm for logic and django-orm for migrations.... - but I'm not good at processing ast-trees ... How do you think - is it bad idea ? :-)
Alexander
I think it may be overly complex task
Anonymous
Alexander
Probably so
Evgeniy
Hi everyone! Could anybody tell usecases in anyio/asyncio - I have a projects where async code is prefered ? And what about migrations ??? It would be nice to have a support of django-like migrations….
Heavy calculations with a large number of database accesses can be wrapped in a decorator that performs them in a separate thread, turning your function into an asynchronous one
Evgeniy
For an example aioify
Evgeniy
https://github.com/yifeikong/aioify
Permalink Bot
https://github.com/yifeikong/aioify
Permanent link to the yifeikong/aioify project you mentioned. (?)
Evgeniy
You are writing ordinary sinchronous code, but it becomes asyncronous
Evgeniy
This is of course a crutch and a Bicycle with square wheels
Anonymous
Hi, I haven't been here for a while. I'm currently refactoring an ETL application and looking forward to use PonyORM instead of SQLAlchemy. One thing I have used with SA quite often was the instrospection/reflection feature, which I actually miss in PonyORM. Is there a plan to implement this? Peewee has it's own pwiz as well, but does not support Oracle. Regards, Thomas
Anonymous
Yep. Basically the thing pwiz does. SA uses autoload=True, when defining a Table class.
Kyle
For an example aioify
Would this be faster than using tortoise-orm? I'm thinking in giving tortoise a go but the benchmarks aren't good for my case... I do alot of filtering and gets
Evgeniy
I don't recommend doing this
Evgeniy
It's not native way
Kyle
Yeah, doesn't feel right
Kyle
I'll stay with blocking code for now
Matthew
category_counts = list(models.select((m.category, m.category_slug, len(m.product.reviews)) for m in models.ReviewProductUserMapping if m.user == current_user_object()).order_by(-3))
Matthew
this query generates a LEFT JOIN SQL query. If I modify the SQL to use a "normal join", it's 4x faster. is there a way to force pony to use a normal join?
Lucky
Thank you 😊 as I understood - db_session context manager is synchronized operation. Just thought, but if I write simple convertor from ponyorm form to tortoise-orm for logic and django-orm for migrations.... - but I'm not good at processing ast-trees ... How do you think - is it bad idea ? :-)
Basically I was thinking about it as well. I guess the problem lies within pony being a monolith in terms how data is structured. If we could seperate the concerns a bit, I think that it would be able to make a step from the current implementation to modern python, async and maybe also including support for typing. We would need to extract the core components: - query builder based on python ast-trees, with support for classes being marked as being a table entity and part of the query. - calling the databases, and basically running and retrieving queries - Object mapper, instantiating objects from the query result and tracking data changes - session logic, keeping objects around and submitting changes to the database at some point
Alexander
this query generates a LEFT JOIN SQL query. If I modify the SQL to use a "normal join", it's 4x faster. is there a way to force pony to use a normal join?
If you rewrite query with normal join, it is not the same query, as it will not include products with zero reviews
Matthew
that's ok in this case
Alexander
In this query you iterate over all "m" objects, and check len of m.product.reviews collection. It is valid case for collection to have zero size, so Pony needs to take that into account. If it is not necessary to include collections with zero size, you can explicitly loop over reviews and use SQL-like count. Similar to this examples: >>> from pony.orm.examples.university1 import * >>> populate_database() >>> select((g, count(g.students)) for g in Group)[:] SELECT "g"."number", COUNT(DISTINCT "student"."id") FROM "Group" "g" LEFT JOIN "Student" "student" ON "g"."number" = "student"."group" GROUP BY "g"."number" [(Group[101], 3), (Group[102], 4), (Group[103], 0), (Group[104], 0), (Group[105], 0), (Group[106], 0)] >>> select((g, count(s)) for g in Group for s in g.students)[:] SELECT "g"."number", COUNT(DISTINCT "s"."id") FROM "Group" "g", "Student" "s" WHERE "g"."number" = "s"."group" GROUP BY "g"."number" [(Group[101], 3), (Group[102], 4)]
Matthew
that worked great, thank you!
Matthew
list(models.select((m.category, m.category_slug, len(m.product.reviews)) for m in user.review_alerts_mappings))
Matthew
with that new query, changing "COUNT(DISTINCT "review"."id")" to "COUNT("review"."id")" seems to cause a big speed increase for some users, which seems to be from postgres parallelism. Is there a way to force that change with pony?
Alexander
In these specific case, where you only have m->product->reviews relations (all 1:1 or 1:many in same direction), you can use count(*), which represented in Pony as just count() without any argument
Matthew
that works great, thanks again
Volbil
@metaprogrammer ssl certificate expired for docs page
Volbil
👌🏻
Anonymous
Hi, sorry to bother, but I haven't received any answer yet about the automatically reflect a database. Any updates?
Anonymous
Another question: is mysql.connector supported or do you still use pymysql oder mysqldb? The one from MySQL seems to be the defact standard.
Alexander
Hi, sorry to bother, but I haven't received any answer yet about the automatically reflect a database. Any updates?
Hi Thomas! There is an incomplete reflection library, but it is for PostgreSQL https://github.com/abetkin/pony-inspect
Permalink Bot
Alexander
Anonymous
Right now Pony does not support mysql.connector, hope we can add its support in the future
Would be great. MySQLdb had its last update 2014, pymysql had an update this year after 2 years of inactivity. It would be great to have support for a really actively developed driver
Lucky
Can I use any(…) on a boolean in a set? chat.network_members = Set(…) network_member.announce_joins = Required(Bool) any(chat.network_members.announce_joins)
Volbil
Hello, out of curiosity is there a way to specify table CHECK constraint using pony (beside raw query) ?
꧁🦔
Hi all! I havent found a way to supress the insert duplicate error, so here is my workaround: def insert_replace(): from pony.orm.sqlbuilding import SQLBuilder func = SQLBuilder.INSERT SQLBuilder.INSERT = lambda *a, **b: ['REPLACE INTO '] + func(*a, **b)[1:]
Lucky
Lucky
huh
Alejandro
da
I'm new into pony, can you give an example of how to use this?
꧁🦔
she replaces insert into command to replace into, globally
꧁🦔
its a temporary solution, not very beautiful lol
Luis
Hi, Can be add a entity after?
Luis
I have a db but need add entity its posible in the generate_mapping?
Alexander
No, currently all entities should be added before generate_mapping
Luis
Thanks Alex
Lucky
Can I do [r for r in ExternalReferral if r.chat.id == update.message.chat.id].order_by(orm.desc(ExternalReferral.foo)).first() ?
Matthew
that's a list comprehension so ordering it like a pony query wouldnt work
Matthew
use select instead?
Lucky
orm.select(r for r in ExternalReferral if r.chat.id == update.message.chat.id).order_by(orm.desc(ExternalReferral.foo)).first() then?
Alexander
Should work, I think
Matthew
yep
꧁🦔
some monkey patch here
꧁🦔
def query_dicts(): from pony.orm.core import Query Query.to_dicts = lambda self: [row.to_dict() for row in self] query_dicts()
꧁🦔
and use:
꧁🦔
MyModel.select(lambda row: row.key == key).to_dicts()
꧁🦔
list of dicts
Volbil
Hey @metaprogrammer any eta when Pony will support python 3.9?
Volbil
I saw someone created draft PR for setup.py
Volbil
https://github.com/ponyorm/pony/pull/565
Alexander
I hope we can do it next week
Volbil
I hope we can do it next week
Got it, sounds great
Alexander
Got it, sounds great
It's not only about setup.py, adding new python support is also about checking if they didnt change something in bytecode and if so - checking that these upadtes doesnt break pony decompiling. But as I know it's okay for 3.9