Hi, I described some steps here: https://stackoverflow.com/questions/16115713/how-pony-orm-does-its-tricks/16118756#16118756

Wow, thanks.

Congratulations!

Congratulations!

Gracias

Happy New Year!

?

?

?

Hi, I have a bunch of queries like this in my postgres slowlog: 2018-01-01 10:03:38 UTC LOG: duration: 1863.315 ms statement: INSERT INTO "x" ... VALUES ... RETURNING "id"

they all seem to happen in the same time in batches

as far as i can tell, there is nothing special about the insert or the table

could the "returning id" be causing the queries to have to wait for each other?

Should I use db.insert https://docs.ponyorm.com/api_reference.html#Database.insert for this?

I am doing a few million inserts of this object per day

Happy New Year to all! Hi Matthew, I think the long insert time may be caused by waiting for some concurrent transaction to finish, for example if that another long-running transaction updated some index, and insert command needs to update that index too. I don't think RETURNING ID clause can slow down the query. Also, I doubt db.insert method may help here - its benefit is that the object is not created in Python before insert, so we can avoid Python overhead when doing millions of inserts. But in your case the slowdown is happened in database and not in Python

Aha! Almost immediately after this insert, I do a commit(), so it looks like that could be the case

Hi there. Happy new year. This is my last resort, i've been trying for hours to find out how to simply create or update a pony database. i poll data from a game, i want to update the entry if it exists, otherwise update it. i got as far as creating it, but i get one error after another when trying to update it, i get "TypeError: float() argument must be a string or a number" for example, even if the value in the db field IS float and the value i'm writing in there is a hardcoded 1.4 or float(100). if i simply skip that line, the same error happens with every other item

is there anyone who can point me in the right direction?

seriously, this

test = int(player_data.group(1)) db_player.id = test,

fails with

TypeError: Value type for attribute Player.id must be int. Got: <type 'tuple'>

the content od test is 710

test = int(player_data.group(1)) db_player.id = test,

remove , at the end, it makes it tuple

OMG. that's what comes from hours of staring at code and not drinking enough. with all that converting from this to that i totally forgot about those :(

i had the data stored in a dict before i wanted to switch to orm...

it works now. you saved my sanity

THANK YOU ❤️

and sorry for being such a dumbass ^^

No problem :)

Is there an "update if exists, add if not" (like the one in MongoDB) in PonyORM?

Is there an "update if exists, add if not" (like the one in MongoDB) in PonyORM?

At this moment you can emulate it with something like my_object = MyEntity.get(foo=x, bar=y) or MyEntity(foo=x, bar=y)

Nice, thank you

At this moment you can emulate it with something like my_object = MyEntity.get(foo=x, bar=y) or MyEntity(foo=x, bar=y)

However it isn't doing this on a database level, so this can easily fail in multiprocess environments

yes

I see there is some interest in get_or_create (https://github.com/ponyorm/pony/issues/131) and that it somehow relates to upsert? I wrote this a while back but I assume it's wrong - can anyone say why? def get_or_create(PonyEntity, defaults=None, **kwargs): created = False # Try get the object with the kwargs passed in result = PonyEntity.get(**kwargs) if result is None: # Update kwargs with defaults kwargs.update(defaults) result = PonyEntity(**kwargs) created = True return result, created

I think it's because of this:

However it isn't doing this on a database level, so this can easily fail in multiprocess environments

^

race condition

Ah - ok I see. So if I'm not parallelising my accesses to those parts of the DB, the code I wrote should be fine to include in some utils.py?

Yes

?

Thanks

At this moment you can emulate it with something like my_object = MyEntity.get(foo=x, bar=y) or MyEntity(foo=x, bar=y)

I use this code quite often, because I rarely need defaults

This is really a missing feature for such a commonly use case

I agree, we need to add it

As far as I know mysql can do it and so can postgres

I am trying to find the previous sunday from a given date, this code works but seems inelegant, is it possible to do this without a for loop or list comprehension?

for i in range(8): last_sunday = today - datetime.timedelta(days=i) if last_sunday.weekday() == 6 and last_sunday != today: break

Check SO: https://stackoverflow.com/questions/18200530/get-the-last-sunday-and-saturdays-date-in-python

ah the modulus is clever, thanks!

In [4]: select(r for r in Result if between(datetime.date(2010, 1, 1), r.date, datetime.date(2020, 1, 1))).count() ...: ...: SELECT COUNT(*) FROM "result" "r" WHERE '2010-01-01' BETWEEN "r"."date" AND '2020-01-01'

def between(a, x, y): return a <= x <= y

It seems like the order of the arguments does not correlate?

Oops, you are right...

Fixed

In [4]: select(r for r in Result if between(datetime.date(2010, 1, 1), r.date, datetime.date(2020, 1, 1))).count() ...: ...: SELECT COUNT(*) FROM "result" "r" WHERE '2010-01-01' BETWEEN "r"."date" AND '2020-01-01'

between(a, x,y) seems better to me

The best would probably be r.date.between(x,y)

between cannot be method of date, because it used with other types as well. I changed its signature to between(x, a, b), and the python implementation is def between(x, a, b): return a <= x <= b It was already implemented this way inside Python -> SQL translator, so queries work the same way as before

That makes sense thank you

Hi! I am learning to program. I asked ru.stackoverflow. You reply in Russian?)) https://goo.gl/t7Ef3y

I can reply in a hour

It is not urgent! Thank you so much!!!

Is there any reason to choose psycopg2cffi for a new project?

Is there any reason to choose psycopg2cffi for a new project?

If I understand correctly psycopg2cffi is useful when using PyPy

hi :)

Actually why is r.date in range(x,y) not supported?

hi :)

Welcome!

Actually why is r.date in range(x,y) not supported?

Because it doesn't work correctly in Python

TypeError: range() integer end argument expected, got datetime.date.

You can do a list comprehension, producing a "range" of datetime objects, which will work with Pony, although performance may not be as good as <, >, etc

Maybe it can work with date, but with datetime it is error-prone, because in check will not work if datetime objects differ by second or millisecond

Good point :)

If anyone uses digital ocean, you can save 50% by resizing your existing VMs: https://blog.digitalocean.com/new-droplet-plans/

Hello! Thank you for great product! Is there a way in Pony to refine the query depending on some conditions? It seems not possible to do inside the generator expressions. What I’d like to get: query = "SELECT * FROM users" if is_adult: query += " WHERE age > 18"

is_adult is local variable?

yep

select(u for u in Users if (u.age > 18 if is_adult else True))

But if you want to just modify it, you can do like q = select(u for u in User) if is_adult: q = q.filter(lambda u: u.age > 18)

Second way seems better, cause produces better SQL. Just same SQL that you want.

That’s it! Thank you

No problem.

I have pickled a python object which includes Pony instances, for example Product[123]

Product[123] has since been deleted, is there a way in python to detect that the object no longer exists, without throwing an exception?

In [21]: models.UserProduct[196416].product Out[21]: Product[41014804] In [22]: models.UserProduct[196416].product.to_dict() —------------------------------------------------------------------------- UnrepeatableReadError Traceback (most recent call last) <ipython-input-22-48cfb4a2236a> in <module?) —--> 1 models.UserProduct[196416].product.to_dict() /root/app/.env/local/lib/python2.7/site-packages/pony/orm/core.pyc in to_dict(obj, only, exclude, with_collections, with_lazy, related_objects) /root/app/.env/local/lib/python2.7/site-packages/pony/utils/utils.pyc in cut_traceback(func, *args, **kwargs) 74 module_name = tb.tb_frame.f_globals.get('name') or '' 75 if module_name.startswith('pony.utils') and tb.tb_frame.f_code.co_name == 'throw': —-> 76 reraise(exc_type, exc, last_pony_tb) 77 reraise(exc_type, exc, full_tb) 78 finally: /root/app/.env/local/lib/python2.7/site-packages/pony/orm/core.pyc in _load_(obj) 4426 objects = entity._fetch_objects(cursor, attr_offsets) 4427 if obj not in objects: throw(UnrepeatableReadError, -> 4428 'Phantom object %s disappeared' % safe_repr(obj)) 4429 @cut_traceback 4430 def load(obj, *attrs): /root/app/.env/local/lib/python2.7/site-packages/pony/utils/utils.pyc in throw(exc_type, *args, **kwargs) 98 raise exc 99 else: —> 100 raise exc # Set "pony.options.CUT_TRACEBACK = False" to see full traceback 101 finally: del exc 102 UnrepeatableReadError: Phantom object Product[41014804] disappeared

So how can I tell that the product is phantom, without throwing an exception?

I'm not sure it is possible at this moment, maybe we need to add something to API