Alexander
Hello, Alexander! I want to ask your opinion as co-founder of Pony: what is the best approach to implement multi-language in DB architecture using Pony?
Suppose I'm designing a diagram of some CMS, and I have a Document entity with following attributes: title, description, content (omit all unnecessary). All of them should be represented in all languages defined in the system (via Language entity).
I see several approaches:
1) To use Json type for such columns, i.e. Document[1].title = {'ru': 'Тестовый документ', 'en': 'Test document'}
Pros:
- easy to retrieve data w/o joining additional tables
Cons:
- might not be suitable for content which can store too large data
2.1) To use additional entity like String which related to Language (1:M relation type) and to all of the fields of Document via M:1, i.e.:
Document(title=[String(lang='en', value='Test document'), String(lang='ru', value='Тестовый документ')])
Pros:
- seems it is most flexible way
Cons:
- the String entity should contain back-refs to each field of Document (title, description, content). But what if we need to localize another entities, like Template, Chunk, Property etc? I think String definition will be too messy.
- more complex select with join, like select((d, t) for d in Document for t in Translation if t.language.code == 'en')
- not quite convenient way to access localized strings as in (1)
2.2) To use intermediary entity Translation which encapsulates all of the fields to be localized. Maybe also to use inheritance for particular entities, like DocumentTranslation, PropertyTranslation etc which will add specific attributes if it needed.
Pros:
- organize messy code of abstract String entity
Cons:
- the same as in (2.1)
3) To use hardcoded attributes like title_ru, title_en, content_ru, content_en.
Pros:
- easy access to attributes w/o joining
Cons:
- not so easy to add new language
Maybe I'm missing something? What can you advice?
Thank you!
It is hard to suggest something specific, as it may depend on what is the most typical scenarios. Like, do you have some "primary" language or not. Maybe I'd use something like:
class TranslationsMixin:
DEFAULT_LANG = 'en'
def setfield(self, attrname, lang, value):
if lang == self.DEFAULT_LANG:
setattr(self, attrname, value)
else:
if not self.info:
self.translations = {}
self.translations.setdefault(lang, {})[attrname] = value
def getfield(self, attrname, lang):
if lang = self.DEFAULT_LANG:
return getattr(self, attrname)
else:
return self.translations.get(lang, {}).get(attrname)
class Person(db.Entity, TranslationsMixin):
first_name = Required(str)
last_name = Required(str)
age = Required(int)
translations = Required(Json)
Anonymous
Hi everyone! Could anybody tell usecases in anyio/asyncio - I have a projects where async code is prefered ? And what about migrations ??? It would be nice to have a support of django-like migrations….
Alexander
Hi! PonyORM does not support async at this moment. For async applications I suggest you to use something like TortoiseORM instead
https://github.com/tortoise/tortoise-orm
Currently work on migrations are paused, I hope to resume it a bit later
Permalink Bot
Alexander
You need to be careful then:
- only use db_session context manager, not db_session decorator with async code
- never call async functions (await foo()) inside db_session
Alexander
I think it may be overly complex task
Anonymous
Evgeniy
Alexander
Probably so
Evgeniy
Evgeniy
For an example aioify
Evgeniy
https://github.com/yifeikong/aioify
Evgeniy
You are writing ordinary sinchronous code, but it becomes asyncronous
Evgeniy
This is of course a crutch and a Bicycle with square wheels
Anonymous
Hi, I haven't been here for a while. I'm currently refactoring an ETL application and looking forward to use PonyORM instead of SQLAlchemy. One thing I have used with SA quite often was the instrospection/reflection feature, which I actually miss in PonyORM. Is there a plan to implement this? Peewee has it's own pwiz as well, but does not support Oracle. Regards, Thomas
Alexander
Anonymous
Yep. Basically the thing pwiz does. SA uses autoload=True, when defining a Table class.
Kyle
Kyle
For an example aioify
Would this be faster than using tortoise-orm? I'm thinking in giving tortoise a go but the benchmarks aren't good for my case... I do alot of filtering and gets
Evgeniy
Evgeniy
I don't recommend doing this
Evgeniy
It's not native way
Kyle
Yeah, doesn't feel right
Kyle
I'll stay with blocking code for now
Matthew
category_counts = list(models.select((m.category, m.category_slug, len(m.product.reviews))
for m in models.ReviewProductUserMapping
if m.user == current_user_object()).order_by(-3))
Matthew
this query generates a LEFT JOIN SQL query. If I modify the SQL to use a "normal join", it's 4x faster. is there a way to force pony to use a normal join?
Alexander
Matthew
that's ok in this case
Alexander
In this query you iterate over all "m" objects, and check len of m.product.reviews collection. It is valid case for collection to have zero size, so Pony needs to take that into account.
If it is not necessary to include collections with zero size, you can explicitly loop over reviews and use SQL-like count. Similar to this examples:
>>> from pony.orm.examples.university1 import *
>>> populate_database()
>>> select((g, count(g.students)) for g in Group)[:]
SELECT "g"."number", COUNT(DISTINCT "student"."id")
FROM "Group" "g"
LEFT JOIN "Student" "student"
ON "g"."number" = "student"."group"
GROUP BY "g"."number"
[(Group[101], 3), (Group[102], 4), (Group[103], 0), (Group[104], 0), (Group[105], 0), (Group[106], 0)]
>>> select((g, count(s)) for g in Group for s in g.students)[:]
SELECT "g"."number", COUNT(DISTINCT "s"."id")
FROM "Group" "g", "Student" "s"
WHERE "g"."number" = "s"."group"
GROUP BY "g"."number"
[(Group[101], 3), (Group[102], 4)]
Matthew
that worked great, thank you!
Matthew
list(models.select((m.category, m.category_slug, len(m.product.reviews)) for m in user.review_alerts_mappings))
Matthew
with that new query, changing "COUNT(DISTINCT "review"."id")" to "COUNT("review"."id")" seems to cause a big speed increase for some users, which seems to be from postgres parallelism. Is there a way to force that change with pony?
Alexander
In these specific case, where you only have m->product->reviews relations (all 1:1 or 1:many in same direction), you can use count(*), which represented in Pony as just count() without any argument
Matthew
that works great, thanks again
Volbil
@metaprogrammer ssl certificate expired for docs page
Alexander
Volbil
👌🏻
Anonymous
Hi, sorry to bother, but I haven't received any answer yet about the automatically reflect a database. Any updates?
Anonymous
Another question: is mysql.connector supported or do you still use pymysql oder mysqldb? The one from MySQL seems to be the defact standard.
Permalink Bot
Alexander
Lucky
Can I use any(…) on a boolean in a set?
chat.network_members = Set(…)
network_member.announce_joins = Required(Bool)
any(chat.network_members.announce_joins)
Volbil
Hello, out of curiosity is there a way to specify table CHECK constraint using pony (beside raw query) ?
꧁🦔
Hi all! I havent found a way to supress the insert duplicate error, so here is my workaround:
def insert_replace():
from pony.orm.sqlbuilding import SQLBuilder
func = SQLBuilder.INSERT
SQLBuilder.INSERT = lambda *a, **b: ['REPLACE INTO '] + func(*a, **b)[1:]
Alejandro
Lucky
Lucky
huh
꧁🦔
Alejandro
da
I'm new into pony, can you give an example of how to use this?
꧁🦔
꧁🦔
she replaces insert into command to replace into, globally
꧁🦔
its a temporary solution, not very beautiful lol
Luis
Hi, Can be add a entity after?
Luis
I have a db but need add entity its posible in the generate_mapping?
Alexander
No, currently all entities should be added before generate_mapping
Luis
Thanks Alex
Lucky
Can I do
[r for r in ExternalReferral if r.chat.id == update.message.chat.id].order_by(orm.desc(ExternalReferral.foo)).first()
?
Matthew
that's a list comprehension so ordering it like a pony query wouldnt work
Matthew
use select instead?
Lucky
orm.select(r for r in ExternalReferral if r.chat.id == update.message.chat.id).order_by(orm.desc(ExternalReferral.foo)).first() then?
Alexander
Should work, I think
Matthew
yep
꧁🦔
some monkey patch here
꧁🦔
def query_dicts():
from pony.orm.core import Query
Query.to_dicts = lambda self: [row.to_dict() for row in self]
query_dicts()
꧁🦔
and use:
꧁🦔
MyModel.select(lambda row: row.key == key).to_dicts()
꧁🦔
list of dicts
Volbil
Hey @metaprogrammer any eta when Pony will support python 3.9?
Volbil
I saw someone created draft PR for setup.py
Volbil
https://github.com/ponyorm/pony/pull/565
Alexander
I hope we can do it next week
Volbil
Alexander
Got it, sounds great
It's not only about setup.py, adding new python support is also about checking if they didnt change something in bytecode and if so - checking that these upadtes doesnt break pony decompiling.
But as I know it's okay for 3.9
Volbil