Jim
alexander y ou mean this is not valid python since it's awaited inside the with db_session ?
Jim
I forgot a line : loop.run_until_complete(do_async(db_use)
Alexander
The first should work, if not - we need to fix the bug The second is incorrect. It will work, but with incorrect result, as with cannot detect async call isnide
Jim
ok. I went further. First of all I was wrong in first example (calling twice the coroutine function). But it might still be a bug. we can't return value from coroutine decorated by db_session. As well I do not know if it's important but decorated with db_session async function are not seen as coroutine function. Here is a complete example import asyncio from pony.orm import Database, Required, db_session db = Database() class People(db.Entity): name = Required(str) db.bind(provider="sqlite", filename=":memory:") db.generate_mapping(create_tables=True) with db_session(): People(name="john") async def dummy_coro(): print("inside dummy coro") return "dummy" @db_session async def db_session_coro(): print("inside db_session coro") return "dummy" @db_session async def db_session_coro_with_query(): print("inside db_session coro with query") pp = People[1].to_dict() print(pp) return pp async def do_async(func): res = await func() print("result in do_async: ", res) return res def main(): loop = asyncio.get_event_loop() print(asyncio.iscoroutinefunction(dummy_coro)) # -> True print(asyncio.iscoroutinefunction(db_session_coro)) # -> False print(loop.run_until_complete(dummy_coro())) # -> inside dummy coro # -> dummy print(loop.run_until_complete(db_session_coro())) # -> inside dummy coro # -> None print(loop.run_until_complete(db_session_coro_with_query())) # -> inside dummy coro with query # -> {'id': 1, 'name': 'john'} # -> None loop.run_until_complete(do_async(dummy_coro)) # -> inside dummy coro # -> result in do_async: dummy loop.run_until_complete(do_async(db_session_coro)) # -> inside dummy coro # -> result in do_async: None loop.run_until_complete(do_async(db_session_coro_with_query)) # -> inside dummy coro with query # -> {'id': 1, 'name': 'john'} # -> result in do_async: None if __name__ == "__main__": main()
Alexander
I hope I would be able to look into it tomorrow
Jim
nothing urgent stay in week end :-)
Jim
Running a query on entity People via asyncio : loop.run_in_executor (that's to say in another thread) fails with pony.orm.dbapiprovider.OperationalError: no such table: People. Is that the expected behavior ?
Jim
the code : import asyncio from pony.orm import Database, Required, db_session db = Database() class People(db.Entity): name = Required(str) db.bind(provider="sqlite", filename=":memory:") db.generate_mapping(create_tables=True) with db_session(): People(name="john") @db_session def simple(): print("inside simple") pp = People[1].to_dict() print(pp) return pp def main(): loop = asyncio.get_event_loop() loop.run_until_complete(loop.run_in_executor(None, simple)) if __name__ == "__main__": main()
Alexander
No, it looks strange
Jim
ok. I went further. First of all I was wrong in first example (calling twice the coroutine function). But it might still be a bug. we can't return value from coroutine decorated by db_session. As well I do not know if it's important but decorated with db_session async function are not seen as coroutine function. Here is a complete example import asyncio from pony.orm import Database, Required, db_session db = Database() class People(db.Entity): name = Required(str) db.bind(provider="sqlite", filename=":memory:") db.generate_mapping(create_tables=True) with db_session(): People(name="john") async def dummy_coro(): print("inside dummy coro") return "dummy" @db_session async def db_session_coro(): print("inside db_session coro") return "dummy" @db_session async def db_session_coro_with_query(): print("inside db_session coro with query") pp = People[1].to_dict() print(pp) return pp async def do_async(func): res = await func() print("result in do_async: ", res) return res def main(): loop = asyncio.get_event_loop() print(asyncio.iscoroutinefunction(dummy_coro)) # -> True print(asyncio.iscoroutinefunction(db_session_coro)) # -> False print(loop.run_until_complete(dummy_coro())) # -> inside dummy coro # -> dummy print(loop.run_until_complete(db_session_coro())) # -> inside dummy coro # -> None print(loop.run_until_complete(db_session_coro_with_query())) # -> inside dummy coro with query # -> {'id': 1, 'name': 'john'} # -> None loop.run_until_complete(do_async(dummy_coro)) # -> inside dummy coro # -> result in do_async: dummy loop.run_until_complete(do_async(db_session_coro)) # -> inside dummy coro # -> result in do_async: None loop.run_until_complete(do_async(db_session_coro_with_query)) # -> inside dummy coro with query # -> {'id': 1, 'name': 'john'} # -> result in do_async: None if __name__ == "__main__": main()
about it : line 619 in core.py : ```except StopIteration: assert not db2cache_copy and not local.db2cache return```adding ```except StopIteration as err: assert not db2cache_copy and not local.db2cache return err.value```solves the problem af not returning values of coroutine function for me
Jim
concerning my trouble using the framework starlette with db_session decorator : starlette use asyncio.iscoroutinefunction to check if the current view is a standard function or a coroutine. if function it runs it in loop.run_in_executor, if coroutine it awaits it. asyncio.iscoroutinefunction return False for decorated coroutine with db_session. So the framework run the corotine in loop.run_in_executor with error : TypeError: 'generator' object is not callable Adding new_gen_func._is_coroutine = asyncio.coroutines._is_coroutine lne627:core.py just before return new_gen_func solves the probleme. starlette see it as coroutine and runs it as coroutine inside db_session. hope it's clear enough
Anonymous
Can I use an instance method for @db.on_connect?
Alexander
yes
Alexander
hmm, need to check
Anonymous
Ok. Thanks.
Alexander
I think currently you need to use a workaround, something like that: def bind_method(db, method): @db.on_connect def func(database, connection): method(database, connection) bind_method(db, myobj.mymethod)
Alexander
Because inside on_connect we have a check for a specific function type: if isinstance(func, types.FunctionType): ... We need to relax that in order to allow instance methods #todo
Anonymous
Ok. I'll give that workaround a try. It looks like it will work for my use case. Thanks.
Vitaliy
Goog evening Alexander! It seems I found a new bug. Now I trying to reproduce it and will post here.
Alexander
Hi Vitaliy! Ok, will wait for example
Jim
Sorry, but still something to ask 🙈 : async def coro(): async def awaited(x): return People[x] guy = await db_session(awaited)(x) return guy.to_dict()Is the inline call of the decorator valid with pony db_session ?
Alexander
I need to inspect what you have wrote to me previously. At this moment I think that in order to be safe right now you need to use Pony in a purely synchronous style, that is in a fragements of code which don't have async calls. Like that: async def foo(): ... with db_session: # no async calls here ... x = await bar() ... with db_session: # no async calls here too ...
Vitaliy
When pony works with inherited object no matter which class to use to get an object. For example I have an "abstract" entity Server and two children classes VirtualServer and PhysicalServer. So when I use: >>> Server[1] I got: PhysicalServer[1] And like this: >>> Server[345] VirtualServer[345] I have a Payment entity which have a Set of Servers. When we iterate servers over some Payment object we will not get the same behavior: >>> for s in Payment[2579].servers: ... s ... Server[548] Once again: >>> s Server[548] But just touch it like this: >>> Server[548] and we got what expected: VirtualServer[548] After this trick it works OK (due to a cache): >>> for s in Payment[2579].servers: ... s ... VirtualServer[548]
Alexander
Thanks for reporting, I'll check this
Vitaliy
However, it works right if do the following (use `select()`): >>> for s in Payment[2579].servers.select(): ... s ... VirtualServer[548]
Vitaliy
In first case here are SQLs: GET CONNECTION FROM THE LOCAL POOL SWITCH TO AUTOCOMMIT MODE SELECT "id", "paid", "created", "updated", "type", "amount", "term", "metadata", "method", "rate", "user" FROM "payment" WHERE "id" = %(p1)s {'p1':2579} SELECT "T1"."server" FROM "payment_server" "T1" WHERE "T1"."payment" = %(p1)s {'p1':2579}
Vitaliy
But in the seconds case are: SWITCH TO AUTOCOMMIT MODE SELECT "id", "paid", "created", "updated", "type", "amount", "term", "metadata", "method", "rate", "user" FROM "payment" WHERE "id" = %(p1)s {'p1':2579} SELECT DISTINCT "s"."id", "s"."type", "s"."name", ….blablabla.... "s"."hypervisor", "s"."uuid", "s"."vmid" FROM "server" "s", "payment_server" "t-1" WHERE "s"."type" IN ('Virtual', 'Physical', 'Abstract') AND "t-1"."payment" = %(p1)s AND "s"."id" = "t-1"."server" {'p1':2579}
Alexander
Originally the object accessed via a foreign key has Server class, because until we retrieve its classtype attribute we don't know its real type. But when user tries to get direct access to that object, Pony forces loading of classtype attribute and fixes object class dynamically by changing it from Server to VirtualServer. It seems that this check is missing when iterating over collection items
Alexander
When you do .servers.select() it leads to a new query which loads classtype attribute before user has access to object, and so it has opportunity to fix object class
Vitaliy
Thank you for detailed description. Will it be changed? Currently I inspect a code of my app to make such workarounds, but shoud I? 🙂
Alexander
Yes, I'll fix it soon, maybe tomorrow
Alexander
It doesn't look like a complex fix
Vitaliy
Ben
Has something changed with the Decimal behaviour in 0.7.7? I'm getting strange issue where my numbers seem to be converted to float and then back to Decimal. e.g. I'm storing Decimal('1.00') into my database, and when I retreive it and print it I get '1'. Any idea of what could cause have caused this change in the latest update?
Alexander
Yes, there was a bug, please upgrade to 0.7.9
Ben
Ooh thanks!
Ben
noted
Ben
I don't see the release note for 0.7.9 on the blog
Alexander
Yeah, my bad
Ben
Ah no worries, just found it on the GitHub
Ben
Thanks a lot!
Alexander
Sure
Ben
I think I found a query compilation error as well
Ben
I get this error: "argument of WHERE must be type boolean, not type bigint at character 140"
Ben
My code: clients = select(e for e in Entrepreneur) # Is sometimes filtered, but in the case of the crash is just like that issues = select(I for I in Issues if I.client in clients) open_issues = issues.filter(lambda I: I.status != 4 and I.status != 5 and I.status != 6)
Ben
The SQL debug: STATEMENT: SELECT COUNT(DISTINCT "i"."id") FROM "issue" "i" WHERE "i"."affectedclient" IN ( SELECT "e"."id" FROM "entrepreneur" "e" WHERE "e"."id" ) AND "i"."status" <> 5 AND "i"."status" <> 6 AND "i"."status" <> 4
Ben
It looks like the SQL output from pony is missing a pair of parentheses
Ben
using pony 0.7.9, but had same issue with 0.7.7
Alexander
Could you open an issue please? https://github.com/ponyorm/pony/issues
Ben
Just did: https://github.com/ponyorm/pony/issues/417
Genesis Shards Community
how do I add a date type field?
Genesis Shards Community
helpme, please!
Rozen
required(datetime.date) ?
Genesis Shards Community
thanks
Genesis Shards Community
from datetime import date ... field = Required(date)
Genesis Shards Community
@Rozen thanks!
Rozen
:)
Ben
I have a question, I'm doing an order_by with a lambda function and a desc() argument; e.g. clients.order_by(argument) with argument = lambda c: c.type.name
Ben
this works fine
Ben
but when I try to do a descending order I do
Ben
clients.order_by(desc(argument)) and the desc() is not taken into account
Ben
is that by design? is there a proper way to do it?
Alexander
Hi, proper way is clients.order_by(lambda c: desc(c.type.name)) Because inside lambda can be tuple with multiple attributes
Alexander
Maybe we'll support order_by(desc(lambda)) style later
Ben
oh right
Ben
thanks
Ben
it would be a nice feature to have
Ben
😊
Ben
Is it possible that the pagination calculation is done before a filter is applied?
Ben
I'm getting an odd behaviour: I'm getting pages of a non-non filtered list, and its fine, I have 10 items by page
Ben
I then add a filter, and some pages have only 4 items
Ben
Ive checked my code and I cant find what could explain this
Matthew
Could you show your full query?
Ben
clients = select(C for C in Clients if Client.village in villages)
Ben
(villages comes from another query)
Ben
Then I do
Ben
clients.filter(lambda C: C.termination_date is None)
Ben
and then I do
Ben
clients.page(1,10)
Ben
and I occasionally get only 3 or 4 in a page (where without the filter I get 10)