In any

Can you reproduce the error stable?

It happens every run. But I'm not sure I can understand why it happens, so i can't reprodice it on smaller example ;/

I think, to understand the error in need to see your query (package_data.py, line 256)

Traceback shows only part of it

Well it's just ordinary select. I can show it.

It would be helpful

https://pastebin.com/2yanTqsX

But noexam_grade is child of ConfirmationDocumentLegacy

Now I need to know model definition of attributes conf_link.entrant and conf_link.right_type

right_type is just int

And entrant... is too big. I can say that noexam inside ConfirmationDocument

So entrant is an entity instance

yes

Actually I can also say that I repeat a lot of actions with db (build dict to send it as json to some service for statistics). And if i do this with a single similar piece of data it won't fail

Maybe something with caches?

What version of Pony do you use?

0.7.2

But... hm. Let me check something.

In version 0.7.2 code lines have different numbers

Yes, maybe it's master branch just before 0.7.2

I will check

I suspect it may be some old commit. I remember that some time ago we fixed similar bug, maybe it is already fixed in 0.7.2

https://pastebin.com/0KkRc6JT

Here

With right line numbers)

Now it is 0.7.2

In pony.orm.core, in line 2126 you can see: if new_dbval is not NOT_LOADED: reverse.db_set(old_dbval, NOT_LOADED, is_reverse_call=True)

That code means the following:

Pony thinks that the attr of obj has different value then before. Previously it was old_dbval and now it equal to new_dbval. Then Pony checks if application code already read previous value of attr attribute of obj instance, and if so, throws the error that you encountered. In order to understand the reason of the error, you can print the values of old_dbval and new_dbval. You can change the line 2126 to the following code: if new_dbval is not NOT_LOADED: log_orm('Value of %r attribute for %r object was changed in database. old: %r, new: %r' % (attr, obj, old_dbval, new_dbval)) reverse.db_set(old_dbval, NOT_LOADED, is_reverse_call=True) Maybe this information can help to debug the problem

Ok, I'll try

Oh my god

I uset Optional instead of Set

I think the error message could be more clear

Sure, I'll think about the error message

Thanks for help. Maybe I'll provide pr for such situation.

*pull request

Hi guys, is there a way to create the database if it does not exist? from pony.orm import * from config import * db = Database() try: db.bind(provider='mysql', host=URL, port=PORT, user=USER, passwd=PASSWORD, db=DATABASE) except OperationalError as e: # Create database if it does not exist print(e)

Hi Juan, we support it for SQLite only. For other DBMSes you typically need to connect to server under admin privileges, create a new user, set permissions, etc. Probably application code shouldn't do this, because it may be security hole to know admin login from application code

Oh, nice, it makes sense. I will create the DB manually. Thank you ❤️

yeah you usually want that as part of your migration/deploy flow

Is there a way to use TEXT or MEDIUMTEXT or there are only the ones listed here? https://docs.ponyorm.com/api_reference.html#attribute-types

You can specify sql_type option: description = Required(str, sql_type='MEDIUMTEXT')

Oh, nice, thank you again :D

Hello guys, I’m checking one issue that I face ocasionally when I query the DB (I guess it’s easier to reproduce during long periods between queries) The exception triggered is RollbackException: InterfaceError: (0, ‘’) The method uses only the orm.get / orm.select methods. I tried to workarround the issues setting @db_session(retry=3) and I still reproduce the issues. So, I would like to know which options I have to fix this issue, and which would be the neat way to do it.

Hi Jordi! What Pony version do you use?

https://github.com/ponyorm/pony/issues/277 Jordi, check this issue. It can help.

0.7.1

We fixed this problem in 0.7.2, now you should see correct error message

Mariadb?

I'm using mysql, I'm going to update the library and recheck. Thanks

After that, check the issue. I think it's your case.

Hi, I have a query for scheduling some work, that I want to improve, so that there are only a maximum of three jobs per user in the results of the query. Is it possible to do this only in pony, with no extra python code to filter results? Assuming this model: Class Job(db.Entity): user = Required(User)

Some users could have no jobs, some could have more than 3 jobs

I'd initially select users with jobs, and then for each user with job select at most three jobs. I'm not sure it is important to pack this into a single query

Yeah it's no problem, just wondered if it was possible :)

I think it is possible in PostgreSQL to write such a query using windows functions, but currently you can express it in Pony only with raw sql

How come limit() isn't valid on a subquery?

like select(u, select(j for j in Job if j.user == u).limit(3)) for u in User)

As I remember, SQL doesn't allow limit N in subqueries

Result table consists of cells, and each cell should contain a single value only

If only you don't write hierarchical query in some database-specific syntax

A nice side effect of the 0.7.2 db_session bug fix, when tweaking a script to use a db_session per intensive set of queries, it's a lot faster, as well as using a lot less memory

> 10x faster

No idea why, but I like it :)

That's interesting :)

Maybe there is an O(n^2) or similar somewhere deep in pony?

that is no longer having a big n

it is a background cronjob that takes a "job", then does a bunch of calculations (so lots of pony queries) to produce a string which is cached in redis

the jobs are independent of each other

Maybe in 0.7.2 we made work easier for Python garbage collector

I imagine it's less work for all layers, going from max 8gb ram usage to ~140mb constant usage

I imagine reusing assigned memory is quicker than asking the kernel for new memory?

I think earlier in some usage scenarios all previously cached objects stay in memory forever, and each garbage collection cycle the garbage collector was needed to re-check all that growing set of objects in order to check they are still cannot be collected. Now only objects from the last db_session need to be collected

That makes sense

:)

I am getting pony.orm.core.OptimisticCheckError: Object X[29] was updated outside of current transaction errors in a scheduler function, which looks like this: @models.db_session def some_scheduler(): for x in pony_select_query: redis_enqueue(x.id) x.scheduled = datetime.datetime.now() If I changed it to the following, would the pony portion successfully retry up to the commit, meaning that once it reaches the redis part of the code, there can't be any OptimisticCheckErrors? @models.db_session(retry=3) def some_scheduler(): x_ids = [] for x in pony_select_query: x_ids.append(x.id) x.scheduled = datetime.datetime.now() models.commit() for x_id in x_ids: redis_enqueue(x_id) I now see in the docs that with retry, commit() shouldn't be explicitly used, is there a way around this? Thanks

OptimisticCheckError arises because several different processes are trying to redis_enqueue the same object. Then, when it's time to update the database, second process sees that the object was already enqueued and its scheduled time already changed. So, the question is, is it OK for object to be enqueued several times? If the answer is yes, you can define scheduled attribute as volatile and Pony will stop consider it as an error. On the other hand, if an object should be scheduled only once, you can use SELECT FOR UPDATE query: query = query.for_update() In that case, a db_session will lock object in the database when selecting it, and other parallel db_sessions will wait until the first db_session finished. I think it may be more efficient to mark scheduled attribute as volatile and then check an object upon retrieving it from the redis queue in order to see if the object was already processed.

I am only ever running the function once at a time, I think it's other code that is changing the object. redis_enqueue should only ever be run once in a given time period (hours). Can I use for_update() with my original code, or something like my latter code?

I think attributes other than scheduled are being changed, so not sure making scheduled volitile would help?