Евгений
Можно ли использовать PonyORM без предварительного задания моделей? Типа select(o for o in Table('MyTable'))
Нашёл только вариант db.select, но это уже чистый SQL - не совсем то, что хотелось бы...
Alexander
ORM works with defined entities in general
Евгений
ORM works with defined entities in general
Почему получаю вот такую ошибку pony.orm.core.DBSchemaError: Column 'id' already exists in table 'table1' class Table1(db.Entity): _table_ = 'table1' pid = PrimaryKey(int, column='id') class Table2(db.Entity): _table_ = 'table2' url_id = Optional(int, column='id')
Евгений
ошибка проявляется на db.generate_mapping(check_tables=False)
Евгений
*База уже существует и мне не надо создавать таблицы.
Vladislav
Define entities and try this: db.generate_mapping(create_tables=False)
Евгений
Define entities and try this: db.generate_mapping(create_tables=False)
Так оно по умолчанию итак create_tables=False
Евгений
generate_mapping пытается добавить колонку в таблицу и я не понимаю почему. https://github.com/ponyorm/pony/blob/main/pony/orm/core.py#L1063
Евгений
hm, poetry install package with ignoring requirements.txt
Alexander
any chance you already take a look Alex?
Not yet, should start on Saturday
Sameer
Hello, Is it possible to increase the connection pool size with pony orm when using oracle database? I am getting below error - python38/root/usr/local/lib/python3.8/site-packages/pony/orm/dbproviders/oracle.py", line 585, in connect con = pool.cx_pool.acquire() cx_Oracle.DatabaseError: ORA-24459: OCISessionGet() timed out waiting for pool to create new connections
Jeremy
Hey guys! found pony recently and think it is the way to do a python orm .. been programming python for 4 years and cant believe i just recently found this. The community is really sleeping on this project. I watched the youtube vids and wrote my own script to generate pony models off a postgres db for my company and use them in my data scrappers.
Jeremy
Super interested in making a donation to get Enum support
Jeremy
https://github.com/ponyorm/pony/pull/585 seems there is a PR open for it, tho idk the code base well enough to have an opinion on it yet
Jeremy
Super interested in making a donation to get Enum support
just signed up as a donor on the website :)
Alexander
Thank you! I'm adding support of Python 3.12, and after that I can look into Enums
Jeremy
of course. Right now I use sqlalchemy for my async stuff with fastapi, then use pony in my data scrapper crons with apscheduler. I would like to be able to use pony end-to-end.
Jeremy
I also like the thought of ponyjs .. so currently i generate both the pony model and its graphql type in my script .. I could see expanding on this and generating generalized graphql resolvers with strawberry for fastapi based on the pony models .. then using graphql-codegen to generate react hooks on those resolvers (which I do this on my own resolvers now in both node/python) .. I use the urql graphql client in my nextjs app with the generated hooks/components to get end-to-end type safety .. where I think pony plays in on the FE is the client side caching layer, currently if some entity updates then best method to handle is invalidating all caches of queries related to that entity's type then refetching .. my urql client is already schema aware but what if it was pony aware and with generalized generated pony resolvers the cache updating could be designed to handle more complex situations .. even though the client is aware of the graphql schema, its not aware of what a resolver does nor the db schema
Jeremy
theres a project that does similar generations with nestjs in node for typical crud rest api. Uses a decorator with some options for some more control over the behavior. https://github.com/nestjsx/crud .. and some other one that seems to do similar for graphql for nestjs https://doug-martin.github.io/nestjs-query/docs/introduction/example/#create-the-dto
Jeremy
maybe the idea isnt feasible with all things consider, but i think the idea is very kool
Jeremy
note the urql client uses observables like knockout does in ponyjs example
Jeremy
being more immediately realistic, I think my company can contribute by continuing on the work of pony-inspect instead of maintaining my standalone script in our repo to generate models from a db schema. Itll be more my weekend project until I get more devs hired; recruiting isnt easy :/ .. and itd prob be just a focus primarily on maintaining what is needed for my company until I can devote more resources to maintenance more generally, ie primarily python 3.11 and postgres support. BUT its better than just keeping my script private in our repo and working on it there. here is the repo I am started from a clone of pony-inspect; a just a basic reproduction but writes to file, fully type hinted, better cli interface, updated with some more recent python stuff/packages. On the dev branch I am moving away from the string concatenation to generate the source and towards aggregating AST nodes into a module to unparse. https://github.com/LobbyMaticInc/pony-gen
Jeremy
src/pony_gen_v2.py is the script I wrote for postgres before finding pony-inspect and starting this repo
Naelson
Hello everyone, is there any way to use an inactive session or when creating a session, store it and continue using?
Naelson
db oracle
Daniel
Hello everyone, i have a question since there is no database migration, available right now in pony orm, how do i go about making changes after my database is already in product, because i think manually altering the database can be dangerous
Ziya
Make db read-only, take snapshot, make changes, load from snapshot. Then change read-only to previous state, if needed.
aldo
good morning, how may I execute a raw_sql on an after_insert() hook
aldo
self._database_.get_connection().cursor().execute(...)
Naelson
Hello, does anyone know if there is a way to change the database connection to the running application? The user can change the provider, user, password, dsn
Amo
Hello, does anyone know if there is a way to change the database connection to the running application? The user can change the provider, user, password, dsn
You can disconnect and create a new connection. There should be some documentation about removing a connection in unit tests somewhere, this should work here too
Naelson
You can disconnect and create a new connection. There should be some documentation about removing a connection in unit tests somewhere, this should work here too
I'm working with django and graphql, with django-graphene, to get the database and define the entities I have a get_db function that returns the database
Naelson
to declare entities, I always call get_db
Naelson
Now I took the database definition to settings.py, but I get an error when running the server, cannot define entity 'ENTITY': database mapping has already been generated
Naelson
Naelson
If anyone has anything in mind on how to solve this problem, I would be grateful.
Amo
I don't know Django, but maybe this could help http://stackoverflow.com/questions/45848795/ddg#47072232
Amo
Tldr: move the entity definitions inside a function
Jeremy
Hello everyone, i have a question since there is no database migration, available right now in pony orm, how do i go about making changes after my database is already in product, because i think manually altering the database can be dangerous
generate models with sqlalchemy on old db schema, then use alembic with sqlalchemy to generate migration script against new db schema .. i juggle both pony n sqlalchemy .. i prefer pony but need migration/async support
Jeremy
https://pypi.org/project/sqlacodegen-v2/
Naelson
Tldr: move the entity definitions inside a function
got to this, It would be like this?
Amo
got to this, It would be like this?
Does it work? I use a function like shown in the link above to declare all my entities.
Amo
To be more clear: I use one function for all entities
Naelson
I define a function for each entity, in different files
Amo
def define_entities(db): class Foo(db.Entity): ... class Bar(db.Entity): ... def open_database(filename): db = Database() define_entities(db) db.bind('sqlite', filename) db.generate_mapping(create_tables=True) return db
Naelson
I'm more concerned about how to update the oracle db information when receiving a post in the api
Amo
I guess you can just create a new database object with the new credentials
Amo
### file models.py def define_entities(db): class Foo(db.Entity): ... class Bar(db.Entity): ... def open_database(filename): db = Database() define_entities(db) db.bind('sqlite', filename) db.generate_mapping(create_tables=True) return db ### file main.py db1 = open_database('db1.sqlite') db2 = open_database('db2.sqlite') with db_session: foos = select(foo for foo in db1.Foo if <condition>)[:] bars = select(bar for bar in db2.Bar if <condition>)[:]
Amo
I guess python is smart enough to handle something like db = open_database('db1.sqlite') ... Do stuff... .. And later.. db = open_database('db2.sqlite')
Naelson
This is the final way I arrived to achieve the expected result
Naelson
This is where I call the function to update the database if it has all the data to link to the database
Naelson
The print function indicating that it has reached update_db
Naelson
I made a query after making a post to update the database, I received this alert
Naelson
This is the final way I arrived to achieve the expected result
I don't know what I'm doing wrong, maybe everything lol Anyone who can advise?
aldo
it may be easier with text so we might copy and execute it
aldo
good morning, to do a join you may do sql = orm.select( (attach, detail) for attach in Attach for detail in Detail if detail in attach.origin )
aldo
the thing is that I already have attach and detail filters, ¿is there a way to combine them? sql_attach = orm.select(x for x in Attach)..add_filter(...) sql_detail = orm.select(x for x in Detail).add_filter(...) sql_attach.join(sql_detail)
aldo
sql = sql_attach.filter(lambda x: x.origin in sql_detail)
aldo
Suppose you have an entity class MyEntity(db.Entity): id = PrimaryKey(int) title = Required(str) description = Required(str) ... After you created the database and tables, you can execute an additional commands like the following: sql_create_fts_table = """ CREATE VIRTUAL TABLE IF NOT EXISTS FtsIndex USING FTS5 (title, description, content='MyEntity', tokenize='porter unicode61 remove_diacritics 1');""" sql_add_fts_trigger_insert = """ CREATE TRIGGER IF NOT EXISTS fts_ai AFTER INSERT ON MyEntity BEGIN INSERT INTO FtsIndex(rowid, title, description) VALUES (new.rowid, new.title, new.description); END;""" sql_add_fts_trigger_delete = """ CREATE TRIGGER IF NOT EXISTS fts_ad AFTER DELETE ON MyEntity BEGIN DELETE FROM FtsIndex WHERE rowid = old.rowid; END;""" sql_add_fts_trigger_update = """ CREATE TRIGGER IF NOT EXISTS fts_au AFTER UPDATE ON MyEntity BEGIN DELETE FROM FtsIndex WHERE rowid = old.rowid; INSERT INTO FtsIndex(rowid, title) VALUES (new.rowid, new.title, new.description); END;""" with db_session(ddl=True): cursor = self.db.get_connection().cursor() cursor.execute(sql_create_fts_table) cursor.execute(sql_add_fts_trigger_insert) cursor.execute(sql_add_fts_trigger_delete) cursor.execute(sql_add_fts_trigger_update) and then in your query you can do something like fts_query = "foo bar*" select( obj for obj in MyEntity if raw_sql(""" id in SELECT(rowid from FtsIndex WHERE FtsIndex MATCH $fts_query) """) ) Can't say about spatialite, I haven't had experience with it
good morning, worried about the performance of the IN (SELECT ...) part. wondering if it could be replaced by a join somehow
Pedro
Hey there!
Pedro
Can someone link me to the permissions documentation for ponyorm?
Pedro
Wasn't able to find it out there
М
Hello, people! Can somebody help me to understand something? I have a Postgres behind Yandex Odyssey connection pooler. When I try to name connection through db.bind(application_name="MYAPP"), it does not work. When I set name of connection through execution db.execute('set application_name = "MYAPP";') on session start, it works fine inside the session, but when session ends and I open new session, application_name resets to nothing. I can, probably, name it in every session with custom decorator or something, but why it works this way? Isn't that true, that in one thread should be one connectiona in pool, so when I run sessions in one thread, it should be same connection? Just curious. And if someone can think of the reason why Odyssey works this way, it woud be great) Here's code example: from pony.orm import Database, db_session db = Database() db.bind( provider='postgres', user='...', password='...', host='...', database='...', target_session_attrs='read-write', application_name='THIS NAME DO NOT APPEAR', ) db.generate_mapping() with db_session(): print('session') db.execute('set application_name = "THIS NAME DO APPEAR";') cur = db.execute("SELECT application_name FROM pg_stat_activity WHERE application_name like 'THIS%'") print(cur.fetchall()) print('we see "THIS NAME DO APPEAR" in connection name list') with db_session(): print('session') cur = db.execute("SELECT application_name FROM pg_stat_activity WHERE application_name like 'THIS%'") print(cur.fetchall()) print('we do not see name in connection name list')
Mr.
open as readbinary
aldo
good morning, worried about the performance of the IN (SELECT ...) part. wondering if it could be replaced by a join somehow
good morning, I think I found a solution. 1. define the tables def create_main_tables(db): class MyEntity(db.Entity): ... def create_fts_tables(db): class MyEntity_fts(db.Entity): pass # note: empty class 2. connect the database and generate the mappings of the main tables db = orm.Database() create_main_tables(db) db.bind(provider='sqlite', filename=filename, create_db=True) db.generate_mapping(check_tables=True, create_tables=True) 3. execute the raw sql to generate the virtual tables sql_create_fts_table = """ CREATE VIRTUAL TABLE IF NOT EXISTS FtsIndex USING FTS5 (title, description, content='MyEntity', tokenize='porter unicode61 remove_diacritics 1');""" 4. reset the connection and now generate all the mappings db = orm.Database() create_main_tables(db) create_fts_tables(db) db.bind(provider='sqlite', filename=filename, create_db=False) db.generate_mapping(check_tables=False, create_tables=False) #note: no checks or creation 5. query sql = orm.select( x for x in MyEntity for fts in MyEntity_fts if x.id == orm.raw_sql('"fts"."rowid"') and orm.raw_sql('"MyEntity_fts" MATCH $value') ).sort_by(orm.raw_sql('bm25("MyEntity_fts")')
aldo
generated sql SELECT "x"."id" FROM "MyEntity" "x", "MyEntity_fts" "fts" WHERE "x"."id" = "fts"."rowid" AND MyEntity_fts MATCH ? ORDER BY bm25("MyEntity_fts")
М
Hello, people! Can somebody help me to understand something? I have a Postgres behind Yandex Odyssey connection pooler. When I try to name connection through db.bind(application_name="MYAPP"), it does not work. When I set name of connection through execution db.execute('set application_name = "MYAPP";') on session start, it works fine inside the session, but when session ends and I open new session, application_name resets to nothing. I can, probably, name it in every session with custom decorator or something, but why it works this way? Isn't that true, that in one thread should be one connectiona in pool, so when I run sessions in one thread, it should be same connection? Just curious. And if someone can think of the reason why Odyssey works this way, it woud be great) Here's code example: from pony.orm import Database, db_session db = Database() db.bind( provider='postgres', user='...', password='...', host='...', database='...', target_session_attrs='read-write', application_name='THIS NAME DO NOT APPEAR', ) db.generate_mapping() with db_session(): print('session') db.execute('set application_name = "THIS NAME DO APPEAR";') cur = db.execute("SELECT application_name FROM pg_stat_activity WHERE application_name like 'THIS%'") print(cur.fetchall()) print('we see "THIS NAME DO APPEAR" in connection name list') with db_session(): print('session') cur = db.execute("SELECT application_name FROM pg_stat_activity WHERE application_name like 'THIS%'") print(cur.fetchall()) print('we do not see name in connection name list')
Hello again. Looks like execution of "DISCARD ALL" request under the hood of Pony when session closes resets application_name. Is it intended behavior or should it be fixed?
Yuri
hello, Is this project dead?
Alexander
Hi! It is not dead, but I couldn't work on it recently due to personal circumstances. I should return to work on it soon.
Yuri
No problem, I hope everything is fine. It's a big project, thank tou.
Denis
Will there be a migration system in pony orm?
Asdf
Will there be a migration system in pony orm?
https://blog.ponyorm.org/2019/03/19/pony-orm-migrations/
Chris
Hi, does anyone know how I can get a PDF version of the docs?