Matthew
what is the to_dict function?
Alexander
Hi all! I wrote backend rest - fastapi - pony app on python 3.8 and i got exceptions in this block of code - TypeError: Entity instance or a sequence of instances expected. Got: <pony.orm.core.QueryResultIterator object at 0x10560e150>
It seems that you are using serialization module. It was an experimental module which is currently not supported. Try to use Entity.to_dict() method for serialization instead
Michele
Hi, is this right? I have a table "Permissions" which will contain a list of permissions (without user association). And then I need to associate 1 or more permissions (from the Permissions table) to a user
Volbil
There is no need to create dedicated collection for that
Volbil
Pony can handle everything for you
Volbil
Just make Set field for user with permissions inside
Volbil
And Pony will automatically create table for that
Michele
Oh right, i don't know what I was thinking, thank you
Jim
Hi, happy new year, I'm struggling with a migration in sqlite using pony : PRAGMA foreign_keys = OFF; DROP INDEX idx_annee__user; CREATE TABLE AnneeBackup (id INTEGER NOT NULL PRIMARY KEY, niveau TEXT NOT NULL); INSERT INTO AnneeBackup SELECT id, niveau from Annee; DROP TABLE Annee; DROP TABLE Utilisateur; ALTER TABLE AnneeBackup RENAME TO Annee; PRAGMA foreign_keys = ON;This works using the python sqlite 3 module (loop line with cursor.execute(line)) but fails (dropping almost everything) with pony looping with db.execute(line) inside a db_session. Any Idea on that ?
Alexander
Hi! I'm not sure it is related to Pony. Are you sure you connect to the same database file?
Jim
Yes. with db_session: for mig in mig_list: self.db.execute(mig)it fails filename = self.db.provider.pool.filename with closing(sqlite3.connect(filename)) as con: with con: for mig in mig_list: con.execute(mig)it works
Alexander
Do you have any exception?
Jim
no
Alexander
Just as an experiment you can change self.db.execute(mig) to self.db.get_connection().execute(mig) To check if it works or not
Jim
no it doesn't work
Alexander
I still think you have something different - database file or migration list
Alexander
Maybe you can print migrations before sending them to self.db.execute to check that they are actually present and the list is not empty
Alexander
Another possibility - do you have some breakpoint later before exiting from db_session? If you check database structure in a separate console, you will not see changes until commit
Alexander
When you do sqlite3.connect directly it will work in autocommit mode by default, so changes will be visible to another connections after each command
Jim
It's the same code just commented to try one or another so it's same migrations. I' tried adding : assert con.execute("select id from Page").fetchall() and self.db.execute("select id from Page").fetchall() after each execute(mig). It appears that if fails with pony at the DROP TABLE Annee; line. Isn't pony also in autocommit mode ?
Alexander
No, Pony uses transactions.
Alexander
> It appears that if fails with pony at the DROP TABLE Annee; line If it fails, you should be able to see an exception
Alexander
Can you please add ddl=True option to db_session?
Jim
it works ! what is this undocumented ddl option ?
Alexander
Hmm, it is undocumented indeed #todo. It mark db session as peforming some DDL (that is, ALTER TABLE) commands. For SQLite it means that db_session turns off foreign key checks
Jim
ok thank you for your help Alexander
Anonymous
Hi all ! Just wanna say that I love Pony and I'm currently using it with FastAPI. That's all!
Jim
Sync and async ?
Anonymous
Maybe someone would be able to help me. I'm trying to update a User instance using the set method. try: user.set(**data.dict()) except Exception as e: raise HTTPException(status_code=404, detail=str(e)) I have a unique constraint in the email and username field so theoretically the Exception should contain such errors whenever users try to update but it already exists.
Anonymous
but I could not catch the exception, here's the traceback
Anonymous
File "d:\projects\csuite\csbackend\.venv\src\pony\pony\orm\dbapiprovider.py", line 67, in wrap_dbapi_exceptions except dbapi_module.IntegrityError as e: raise IntegrityError(e) pony.orm.dbapiprovider.IntegrityError: UNIQUE constraint failed: user.email I also tried catching the pony.orm.dbapiprovider.IntegrityError
Anonymous
data = Pydantic model containing the email and username fields.
D
Seems you try to add user with duplicate email, while you set unique property in pony model
Anonymous
Yup I'm trying to catch the IntegrityError but the try except apparently cannot catch it.
Anonymous
Alright I kinda get it now. Since the transactions are only saved on function end, the exception could not catch it since technically it didn't throw it that time. The transaction was committed when the function / view returned a response. Fixed it by calling commit in the try instead of the set.
bigboy
Hey guys, For my bachelor thesis I try to reproduce the SQL-Statement in the link. I use the TPC-H database, it is a benchmark database and the manufacturer gives examples of SQL Statements which I try do build with ponyORM. At this point I am not able to build this Queries and I am getting to the point where I ask myself is this kind of Query possible to create with ponyORM. I know it’s more complex than other SQL Statements, so I would be really thankful if someone could give me an answer: • Is it possible to build it with ponyORM? • Could someone help me to build it? Thanks in advance https://pastebin.com/T1rww1Bd
Alexander
Hi! Yes, it is possible. At first you need to define entities and relationships between then, and then you would be able to use relationships to express this query
Alexander
You can try to define entities and show them here
Alexander
Actually, the query does not look particularly complex
bigboy
Actually, the query does not look particularly complex
After reading the docs everything worked as it should!
Maik
hello guys, is there a way to have a unique combine field in the db? so the combination of two entities need to be unique?
Alexander
class MyEntity(db.Entity): foo = Required(str) bar = Required(int) composite_key(foo, bar) https://docs.ponyorm.org/entities.html#composite-keys
Maik
thanks @metaprogrammer didnt know it called that way
Maik
i searched on the webite to unique text but it doesnt show that object... thats why i couldnt fine it (the UNIQUE is in the text)
Alexander
Thank you for reporting, maybe we rephrase it #todo
Sergey
Hello, how is there a field with the buffer type how to load a file into it
Alexander
are you using Python 3?
Sergey
Yes, Python 3.8.4
Alexander
with open('myfile') as f: data = f.read() obj.field = data
Volbil
What is the point storing files in db?
Volbil
Just curious
Alexander
What is the point storing files in db?
- it may be easier to achieve consistency. If database field stores filename only, then the file can be removed from the filesystem and the database will not know about it. - it is easier to copy entire database to another machine this way - it requires less efforts comparing to maintaining correct directory structure But it has drawbacks as well, so this is a tradeoff
Lucky
Is there a way to retrieve only the WHERE clause of a python orm.select(…)? I want to update many at once, here'd be random example I just created for this orm.select(row for row in Table if row.foo == row.bar).update(Table.bar, None) which would be something like UPDATE table AS row WHERE row.foo = row.bar SET row.bar = NULL Or is that kind of batch update already possible, and I missed that somehow?
Matthew
I don’t think update statements are possible yet
Lucky
Yeah, which is really a bummer for stuff like database migrations or even some regular use.
Lucky
Can one somehow tap into the decoded select WHERE statement, so I can build the .update(Table.bar, None) myself?
Anonymous
Hi, @metaprogrammer . i get this error DatabaseSessionIsOver when i try to access print(bool(user.cars)) but i don't get it when i try for example print(bool(user.language)). my model is like class User(db.Entrity): ... cars = Set("car") class Car(db.Entity): ... owner = Required(User)
Anonymous
i have to say, it doesn't happen when i just print(user.cars). it happens only with print(bool(user.cars))
Alexander
Hi, @metaprogrammer . i get this error DatabaseSessionIsOver when i try to access print(bool(user.cars)) but i don't get it when i try for example print(bool(user.language)). my model is like class User(db.Entrity): ... cars = Set("car") class Car(db.Entity): ... owner = Required(User)
It is expected if you are trying to access the content of user.cars outside of the original db_session. user.cars is a relationship and needs additional SQL query to be loaded, and queries are possible inside db_session only. It is better to move all code that works with ORM objects into the db_session (including these prints). If it is impossible for some reason, as a workaround you can access user.cars inside db_session. Then it will be loaded from the database and cached, and you will be able to access it later.
Anonymous
Thanks for everything, it’s amazing library
Anonymous
Why i could pass it to print() outside the db_session, and It didn’t raise the error? It got raised only when i used an extra func print(f(u.cars))
Alexander
Is something like @db_session def get_relationship(u,r): return getattr(u, r) "Nice"? I mean, whether it's ok or not.
It doesn't look correct. - db_session purpose is to denote transaction boundaries. At the start of db_session Pony grabs the database connection from the connection pool, and at the end of db_session Pony commits the transaction and returns the connection to the pool. - It is possible to have nested db_session, but inner db_sessions are ignored (maybe in the future releases of Pony nested db_sessions will start nested transactions). - Your get_relationship function clearly does not represent an entire transaction, as it receives an u object from the outside. This means that you already have db_session in the outer scope where you retrieved the u object from the database, and the nested db_session around the get_relationship function will be ignored. > Why i could pass it to print() outside the db_session, and It didn’t raise the error? If some object attributes were retrieved during the db_session, their values are still available even after the db_session was finished. As they were already retrieved from the database, they don't need a new query. But it is better to work with all objects inside db_session.
Anonymous
It doesn't look correct. - db_session purpose is to denote transaction boundaries. At the start of db_session Pony grabs the database connection from the connection pool, and at the end of db_session Pony commits the transaction and returns the connection to the pool. - It is possible to have nested db_session, but inner db_sessions are ignored (maybe in the future releases of Pony nested db_sessions will start nested transactions). - Your get_relationship function clearly does not represent an entire transaction, as it receives an u object from the outside. This means that you already have db_session in the outer scope where you retrieved the u object from the database, and the nested db_session around the get_relationship function will be ignored. > Why i could pass it to print() outside the db_session, and It didn’t raise the error? If some object attributes were retrieved during the db_session, their values are still available even after the db_session was finished. As they were already retrieved from the database, they don't need a new query. But it is better to work with all objects inside db_session.
about 3: It isn’t inner, it’s like database/utilities.py# ... @db_session def get_or_insert(self, **properties): user = User.get(**properties) if not user: user = User(**properties) return user project/file.py from ..database.utilities import get_user user = get_or_insert(id=3) And then i can pass ‘user’ to get_relationship in order to get the relationship, and it will happen in its own @db_session, the function doesn’t get called from another db_session-wrapped function.
Alexander
You speak about three different functions, get_user, get_relationship and get_or_insert and don't show how they are interacting
Alexander
1. return just u 2. move db_session up in the stack, so all actions with u are wrapped into this db_session
Anonymous
Alexander
yes, and this is the only db_session that you need
Anonymous
Ah, I thought i could maintain all db-related functions under database/utilities.py# Seems like not good way
Alexander
Also you can apply db_session in the middleware of your web framework to avoid wrapping of each individual endpoint
Anonymous
Btw, are you willing to add get_or_insert in the future?
Alexander
Yes
Anonymous
Oh good