Matthew
I can never remember how to do this in pony - how do I select the object with the largest id for each day for the past n days?
Matthew
select((s.created.date(), max(s.id), s) for s in FacebookDataSnapshot if s.created.date() >= datetime.date.today() - datetime.timedelta(days=7))[:]
Matthew
This works, but then I have the ID in the results, can I do itt anotther way so it only returns the date and the object?
Alexander
I'm no sure this is a correct query
Alexander
Probably it should be something like
Matthew
oh yeah it doesnt give one result per date
Alexander
select(s for s in FacebookDataSnapshot if s.id in select(max_id for date, max_id in ( select((s.created.date(), max(s.id)) for s in FacebookDataSnapshot if s.created.date() >= datetime.date.today() - datetime.timedelta(days=7)) ))).order_by(lambda s: s.created.date)[:]
Alexander
In SQL this query is not an easy one too
Alexander
Note that I removed s from (s.created.date(), max(s.id)) to group objects by date
Matthew
that makes sense! I am getting AttributeError: 'MethodMonad' object has no attribute 'getsql' though
Matthew
ah fixed it
Alexander
I forgot parentheses after the last date
Matthew
removed the date from the order by
Alexander
ok
Matthew
Thank you!
Alexander
Sure
Lucky
What is the current way of supplying a pony orm model as pluggable ORM in libraries, next to e.g. redis or mongodb connectors? https://github.com/ponyorm/pony/issues/330 I don't like to start using sql alchemy just for that.
Alexander
Hi Luckydonald, I answered on GitHub: https://github.com/ponyorm/pony/issues/330#issuecomment-470153594
Luis
Hi 🙋‍♂
Luis
I am try use pony with fdb but show this error no module named fdb
Luis
And import fdb not show nothing error
Luis
I installed fdb
Alexander
Hi, at this moment Pony doesn't work with fdb, it supports PostgreSQL, MySQL, SQLite and Oracle
Luis
😔
Luis
Thanks Alex 👍
Lucky
What's fdb?
Alexander
Firebird
Lucky
Firebird
Interesting. Can someone enlighten me why it would be chosen as DB compared to mysql/postgres?
Lucky
Can't really see any huge differences
Matt
I think it’s mostly used in enterprise. It’s a descent of interbase or something. Think it runs on more commercial unixes and mainframes and stuff compared to Postgres or whatever. Stuff like AIX, Solaris, HP-UX, IBM ZOS...
Matt
Same reasons you might pick innoDB vs another MySQL engine I guess. Some workloads are better suited to it. Don’t ask me which ones tho
Matt
I would guess there is not a lot of overlap between ORM users and fire base users :-P
Felipe
Hi there
Alexander
Welcome!
Lucky
The main benefit of upsert is atomicity. X.get(foo=1) or X(foo=1) sometimes can raise an exception, if another transaction was able to insert object in between
Hey, do we have upsert support now? I don't wanna get UnrepeatableReadError/OptimisticCheckError, but just create/update a entity. Especially in concurrent environments, the last should just win.
Lucky
MongoDB has mongodb_table.replace_one( filter={'chat_id': chat_id, 'user_id': user_id}, replacement={ 'chat_id': chat_id, 'user_id': user_id, 'state': state_name, 'data': state_data, } )
Alexander
Hi! I forget, what database are you using?
Lucky
All of them, the user using the library should decide later.
Alexander
Sorry, we don't have support of upserts yet But I agree this is a important feature Maybe I can add it next weekday
Henri
👍
Lucky
This would be really great. Many SQL languages already have a concept of that, and for the ones that doesn't a regular get + set/create approach could be put in as placeholder, to at least get startet.
Lucky
My current implementation while True: try: # updating existing state first State.get( chat_id=chat_id, user_id=user_id ).set( chat_id=chat_id, user_id=user_id, state=state_name, data=state_data, ) break # successful except Exception: pass # end try try: # creating a new state State( chat_id=chat_id, user_id=user_id, state=state_name, data=state_data, ) break # successful except: pass # end def # end while Most certainly this will get to a point in time where it creates/updates without raising any errors.
Lucky
What would the best current possible way to get around the failures due to concurrency and multiple instances?
Alexander
How often do you need this operation in transactions?
Alexander
I mean, what percentage of transactions uses this operation?
Lucky
I am building a state managment system which will load at a telegram (tele-)flask request, and store the new state afterwards. class State(db.Entity): user_id = Required(int) chat_id = Required(int) state = Required(str) data = Required(Json) orm.PrimaryKey(user_id, chat_id) # end class This particular library (telestate) does only that, but a user could do his own stuff in between. Does that provide answer to your question?
Alexander
Will single db_session mix this update and arbitrary user updates?
Lucky
Hmmm.... Here is a pseudocode mock of how it'd work def process_incomming_telegram_update(self, update): state = self.load_state_for_update(update) for func in userfuncs: global state result = func.process_update(update) telegram.send(result) # end for self.save_state_for_update(update, state) I have a dedicated load_state_for_update and save_state_for_update which I'll probably decorate with @orm.db_session. Therefore I'd have separate sessions to my understanding.
Alexander
Then you may have special entity UpdateLock (probably with a single instance), and in your transaction before getting or creating State instance you can do ul = UpdateLock.select().for_update().first() <call get_or_create> It should fix concurrency problems, but all State updates will be sequential
Lucky
Btw, found related issue #131.
Lucky
For future reference, that's https://github.com/luckydonald/telestate/blob/deploy2/telestate/contrib/pony.py
Permalink Bot
Lucky
Btw, if you wanna get rid of the "user joined" service notifications, you can add @NoServiceBot with deletion permissions
Alexander
We want to detect spam
Lucky
Okey :)
Matt
Does Pony work on top of pg8000 or only psycopg2?
Anonymous
Hi guys!
Anonymous
A quick simple question - should I manage @db_session or just cover my entire codebase with it?
Anonymous
I've created a Stack Overflow question about this with more details, if needed, though https://stackoverflow.com/questions/55168799/should-i-minimize-the-usage-of-db-session-in-pony-orm
Matt
effectively exiting a session (going out of with scope) will commit all your pending transactions to the db
Matt
so maybe that informs the question a little bit
Matt
depends what you want your writes to look like
Matt
ah, I saw your SO question
Matt
well - do you want to block execution of the process until the DB write is completed?
Anonymous
Yes, I understand that, that's why I included an explicit commit() in the contra-example
Anonymous
I am asking rather about the performance/resource implications, for example many concurrent db_sessions might deplete the available connections
Anonymous
But I have a strong feeling that I'm prematurely optimizing :D
Matt
I guess I don’t understand your question beyond that - I guess general practice of scope minimization would apply, e.g., if you don’t have an explicit reason to be inside a block, don’t be
Matt
db_session blocks included?
Anonymous
The question beyond that is the db_session might hold a connection to the database busy (not returning it to the connection pool) I applied this principle, but encountered an issue where pytest-ponyorm assumes db_session everywhere while my code doesn't, so I started to consider adding db_session everyhere to my Flask application as described in the Pony documentation: https://docs.ponyorm.org/integration_with_flask.html#integration-with-flask
Matt
Ah. Don’t have any advice then, sorry
Anonymous
No problem, thank you very much :)
Anonymous
@Jim I'm the author of that PR :))
Jim
Lol
Anonymous
At one point after submitting the PR, I've suddenly realized that I might be introducing an anti-pattern to the use of Pony Maybe all/most of the code should be under the db_session anyways :D As in pony orm documentation they just offer a quick solution to cover all of the Flask endpoints automatically, with no encouragement for covering only the general parts that need it