Is there some place else we could check them?

not really, I saw someone playing with __new__, but that seemed really odd and I didn't trust it

Naah, don't use __new__

thanks

Hi, everybody. I came here just to check if community here as dead as on github. Don't want pony to die. ;D

Message deleted

Hi Xunto! The community is not dead, but it is not as big as for Django. But as a main developer I can assure you that Pony will not die before me :) Last weeks I worked on migrations and missed the bug on GitHub that bothered you. Actually, it was already fixed, you can take development version of Pony from GitHub, and check if it works as expected. On monday I want to release 0.7.2 version of Pony which will include this fix

🎉🎆🎉

@akozlovsky I saw your answer at github. Thanks. But unanswered issue wasn't the main cause of my worry. It's just to quite there for orm that is that good. It's good to hear it's not dead.

Last time releases began to appear less often, it is caused by work on migrations. I hope that after we publish the migration tool, the frequency of releases with new features should increase

https://ponyorm.com/css/img/top-img.png

For some of my pony queries, I have redis caching of the results, this is done on an ad-hoc basis, has anyone thought about having optional caching built in one a per-query basis?

Yes, we have such plans too, but currently other tasks have higher priority for us

It will be great to have such a layer

In cases where all work with database is performing through Pony, it would be possible to have a consistent write-through redis cache, which can completely avoid database hits for some db sessions

Can you give an example session?

https://ponyorm.com/css/img/top-img.png

The main problem with caching layer is with data invalidation. Many queries share the same information. So, when an update is performed, it is hard to tell which query results need to be invalidated. One solutions is to invalidate all queries for specific entity if one instance of entity was changed. Another option is to have limited time-to-live for each query result, and to have a chance to get inconsistent result before that timeout. I have in mind the cache, in which each object is cached and invalidated separately by id. Then query can be automatically transfomed to retrieve only ids from the database, and other fields will be received from Redis. In some cases, such query will be more lightweight. If db session retrieve objects by id or by reference through attributes and collections, then all information can be loaded from Redis

Might it be most simple (and highest ROI) to do something like passing an optional TTL to a certain query, and if no TTL is passed, it's not cached?

For me, it's often only a few queries in a session that are intensive

there isn't too much value in pulling an entity by ID from redis rather than the database

Maybe: select(x for x in X where x.y == 10).cache(ttl=3600)

Sure, it is more simple, but sometimes the query result can be incorrect before TTL. If this is acceptable, then this approach is the most appropriate

Right, if it's a manual tagging of queries, the programmer can know that staleness isn't an issue

there isn't too much value in pulling an entity by ID from redis rather than the database

Yeah, putting in another db seems strange

Can then use machine learning if you want to work out which queries to cache :)

lol

Can then use machine learning if you want to work out which queries to cache :)

Hah, this would be fun at least

One way I get around staleness in my custom caching is having the ability to have a custom caching key. If the pony caching key was the raw SQL of the query, a custom key could be appended, like '{date}' or '{current_user_id}' or {unix_time}'

so the key might be "select * from x where y = 10.2017-07-10"

after midnight, the old cache is no longer used

so maybe query.cache(ttl=86400, key_template='{date}')

One way I get around staleness in my custom caching is having the ability to have a custom caching key. If the pony caching key was the raw SQL of the query, a custom key could be appended, like '{date}' or '{current_user_id}' or {unix_time}'

Are you caching queries or results?

the results of certain queries

https://gist.github.com/db6c492ac320b8608190b2fcf055866f

I do some weird stuff with functools.partial, not proposing that for pony :)

I just use cPickle to store the query results in redis

the gist I gave is probably a bad example, as it is quite complex

https://gist.github.com/anonymous/608bc523c0410ad73b5463581981252e

this seems to work

models.redis is a redis connection object

the query sql could be hashed if memory usage was a concern

It would need to only be used for SELECTs of course

Caching inserts 🤣

Is there a way to access get_sql() on a query that ends in count() ?

since count() is not lazy

Right now - only if you rewrite query as select(count() for x in ...). Also, right after query execution, you can see db.last_sql, but this is a bit too late

since count() is not lazy

Maybe you need to subclass it and implement that yourself?

If it was integrated into pony as query.cache() then I don't think subclassing would work

I think that caching code is at the wrong layer, it needs to be within pony, after sql is generated but before it is executed

Then count(), first(), limit() etc would work

I think some time later we will add such a layer

Hi Guys, We're trying to determine a way to add columns to a table on the fly with pony. for example remap... Table Customers(db.Entity) email = Required(str, unique=True) name = Required(str) ... to: Table Customers(Customers) surname = Required(str)

Is there a clever way to do this?

In order to do this you need a migration tool. I'm working on it right now. It should be ready soon, at first I plan to put experimental version on GitHub, and then we include it in official release

Nice.

In order to do this you need a migration tool. I'm working on it right now. It should be ready soon, at first I plan to put experimental version on GitHub, and then we include it in official release

Also there also is github.com/luckydonald/pony_up

Permanent link to the luckydonald/pony_up project you mentioned. (?)

Hi guys, We are currently working on an upgrade of our Enity-Relationship Editor. Here you can take a look at the current beta version of the interface: https://beta.editor.ponyorm.com/. You can use the same login and password as for the current editor, as it is connected the production database. Your feedback is greately appreciated!

Hi guys, We are currently working on an upgrade of our Enity-Relationship Editor. Here you can take a look at the current beta version of the interface: https://beta.editor.ponyorm.com/. You can use the same login and password as for the current editor, as it is connected the production database. Your feedback is greately appreciated!

Oh gosh, he has a profile picture now. The red M is gone!

Oh gosh, he has a profile picture now. The red M is gone!

😀

Before we release migrations we had to fix some bugs And we are happy to release Pony version 0.7.2!

# Pony ORM Release 0.7.2 (2017-07-17) ## New features * All arguments of db.bind() can be specified as keyword arguments. Previously Pony required the first positional argument which specified the database provider. Now you can pass all the database parameters using the dict: db.bind(**db_params). See https://docs.ponyorm.com/api_reference.html#Database.bind * The optimistic attribute option is added https://docs.ponyorm.com/api_reference.html#cmdoption-arg-optimistic ## Bugfixes * Fixes #219: when a database driver raises an error, sometimes this error was masked by the 'RollbackException: InterfaceError: connection already closed' exception. This happened because on error, Pony tried to rollback transaction, but the connection to the database was already closed and it masked the initial error. Now Pony displays the original error which helps to understand the cause of the problem. * Fixes #276: Memory leak * Fixes the __all__ declaration. Previously IDEs, such as PyCharm, could not understand what is going to be imported by 'from pony.orm import *'. Now it works fine. * Fixes #232: negate check for numeric expressions now checks if value is zero or NULL * Fixes #238, fixes #133: raise TransactionIntegrityError exception instead of AssertionError if obj.collection.create(**kwargs) creates a duplicate object * Fixes #221: issue with unicode json path keys * Fixes bug when discriminator column is used as a part of a primary key * Handle situation when SQLite blob column contains non-binary value

https://blog.ponyorm.com/2017/07/17/pony-orm-release-0-7-2/

@alexeymalashkevich yey cool

Great to have a dictionary for db_bind!

Thank you!

Matthew, I think we fixed the memory leak. You can try to use db_session without strict=True option, it should work without any problem

Happy to test that, is that listed in the above notes?

Yes, > Fixes #276: Memory leak

was it the cache building up objects, or something else?

memory usage seems stable without strict=True :)

The problem was in handling expressions like query = query.filter(lambda obj: obj.field == x) We have some internal function get_lambda_args which returns names of function arguments, like get_lambda_args(lambda obj: obj.field == x) -> ["obj"] That function caches lambdas for speed. But function can have closures with values of external variables, like x in this example. That values become cached too. If x is an instance of object, the function that sitting in a lambda cache holds reference to that object indefinitely. And object has a reference to a whole db_session cache with all loaded objects. As a result, all previously loaded objects stays in the memory. In order to fix that memory leak, now I cache function code object instead of function itself. Codeobject does not linked with any closure, so now all objects can be garbage collected properly

I'm very glad that's fixed!

Me too :)

So big db_sessions still can use up a lot of memory, but it'll be reclaimed upon exiting the session?

or is strict=True still needed in that case?