Micaiah

I want to use pony, but I can't while it still doesn't have migrations.

Should be available in a few weeks

Mikki

Welcome

Evgeniy

i be suprised see telegram group in site))

Mikhail

how about thread safe?

Mikhail

should we expect asyncio/tornado support in near future?

Alexey

should we expect asyncio/tornado support in near future?

in the near future we release Python 3.6 support and migrations asyncio/tornado is not easy feature, we are not planning to start working on it during the next 6 month

Mikhail

it's a main reason I didn't migrate on pony

Anton

In order not to block asyncio ioloop, you can run queries in separate threads. Here is a small example of PonyORM query in aiohttp handler: https://gist.github.com/grudko/4c1dcd2a2eb29bc3d98cd19bfd916cbd

Alexander

Most queries that retrieve objects by primary keys can be run synchronously from asyncio/tornado, and it will be faster than using async protocol

Lucky

In order not to block asyncio ioloop, you can run queries in separate threads. Here is a small example of PonyORM query in aiohttp handler: https://gist.github.com/grudko/4c1dcd2a2eb29bc3d98cd19bfd916cbd

So this part? `async with threadpool():`

Anton

Oh, I used threadpool from https://github.com/agronholm/asyncio_extras

Lucky

Maybe I should learn async

Alexander

The pattern @agrudko suggests may be helpful if db_session contains long queries (analytics, aggregation, etc.)

Alexey

Pony ORM Release 0.7.1: Python 3.6 support and some bugfixes https://blog.ponyorm.com/2017/01/10/pony-orm-release-0-7-1/ yay 😄

Michael

❤

Romet

Nice! Great job.

Lucky

Pony ORM Release 0.7.1: Python 3.6 support and some bugfixes https://blog.ponyorm.com/2017/01/10/pony-orm-release-0-7-1/ yay 😄

Throw an error with more clear explanation when a list comprehension is used inside a query instead of a generator expression: “Use generator expression (… for … in …) instead of list comprehension [… for … in …] inside query” yep, that was me xD

Micaiah

Woohoo!

Micaiah

7.1 doesn't come with migrations correct?

Alexey

7.1 doesn't come with migrations correct?

Correct. 0.8, which we release soon, will

Micaiah

Correct. 0.8, which we release soon, will

Great to hear!

Mark ☢️

see yellow ribbon

Alexey

see yellow ribbon

That's strange

Vasiliy

Hello everybody. I use Pony for connect to the remote Postgres database. Periodically, the database connection is lost and Pony returns an exception: File "... / lib / python2.7 / site-packages / pony / orm / core.py", line 1566, in close try: provider.rollback (connection, cache) File "<auto generated wrapper of rollback () function>", line 2, in rollback File "... / lib / python2.7 / site-packages / pony / orm / dbapiprovider.py", line 59, in wrap_dbapi_exceptions raise InterfaceError (e) RollbackException: InterfaceError: connection already closed After that, Pony automatic reconnect, but my function call is aborted. Is it possible to test the connection and forced to reconnect? I have not found the necessary methods in Pony docs. Maybe you need to look at psycopg2 docs?

Alexander

Hi Vasiliy! Pony should be able to reconnect automatically, if the connection was lost not in the middle of transaction. I think that your problem caused by the following sequense of actions: 1) An application code performs some query; 2) Due to some problem in configuration (for example, an insufficient size of some buffer) the server drops the connection, and Pony gets an exception from psycopg2; 3) Exception unrolling leads to exiting from db_session; 4) Pony tries to handle exit from db_session and attempts to peform rollback, assuming that the connection is stlii active; 5) At this moment we got another exception, "connection already closed" which masks previous exception. I think we need to improve handling of rollback so it does not mask previous exception in case the connection is closed already. Regarding your situation, I think the right way is not to attempt to reconnect, but to prevent an exception which lead to connection closing. After we fix handling of rollback, you should be able to see this original exception and will be able to fix its root

Святослав

Hi. I have a question about ready to use in production. Can i safely upgrade my database schema? I saw migrations not implemtented in before.

Alexander

We are working on migration tool right now. It should be released really soon

Святослав

Without this tool only way it's handmade migration?

Alexander

If you add a new entity or many-to-many relationship, and you call generate_mapping with create_tables=True, Pony will issue CREATE TABLE commands automatically. If you add a new attribute to previous entity, right now you need to perform ALTER TABLE command manually. Typically such commands are pretty simple, but with migration tool it will be easier to write and execute them

Святослав

Thanks. This is what i wanted to hear!

Vasiliy

Hi Vasiliy! Pony should be able to reconnect automatically, if the connection was lost not in the middle of transaction. I think that your problem caused by the following sequense of actions: 1) An application code performs some query; 2) Due to some problem in configuration (for example, an insufficient size of some buffer) the server drops the connection, and Pony gets an exception from psycopg2; 3) Exception unrolling leads to exiting from db_session; 4) Pony tries to handle exit from db_session and attempts to peform rollback, assuming that the connection is stlii active; 5) At this moment we got another exception, "connection already closed" which masks previous exception. I think we need to improve handling of rollback so it does not mask previous exception in case the connection is closed already. Regarding your situation, I think the right way is not to attempt to reconnect, but to prevent an exception which lead to connection closing. After we fix handling of rollback, you should be able to see this original exception and will be able to fix its root

Thanx 4 answer. I found a solution with your help.

Alexander

Vasiliy can you open a new GitHub issue? https://github.com/ponyorm/pony/issues Solving this issue should fix @dadecar problem too

Anonymous

Yes, please! 😊

Vasiliy

Vasiliy can you open a new GitHub issue? https://github.com/ponyorm/pony/issues Solving this issue should fix @dadecar problem too

https://github.com/ponyorm/pony/issues/219

Alexander

Thanks!

Alexander

I think I fixed the bug. Previously the "connection already closed" exception masked the original exception which causes the connection closing. Now you can see the reason for connection closing, and if it is caused by some bug in Pony you can open another issue related to that specific problem.

Alexander

https://github.com/ponyorm/pony/commit/1146ff1f4c3525393d4636406ae41213a628f4b1

Mikki

Welcome :)

Kojo

Thank you

Vasiliy

Thanks, now i receive another exception, your patch is really help me! For example: File ".../pony/orm/dbapiprovider.py", line 55, in wrap_dbapi_exceptions except dbapi_module.DatabaseError as e: raise DatabaseError(e) DatabaseError: server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request.

Alexander

You need to check, maybe your PostgreSQL client and server versions differ: http://stackoverflow.com/questions/15934364/psql-server-closed-the-connection-unexepectedly/15977710#15977710

stsouko

Hello! Is it possible to use fixed-length bit string dtype of postgres in pony?

Alexey

Hello! Is it possible to use fixed-length bit string dtype of postgres in pony?

You can specify the type this way https://docs.ponyorm.com/api_reference.html?highlight=sql_type#cmdoption-arg-sql_type

stsouko

ok. if I specify sql_type('bit(8)') then which type I get in Python?

Alexey

Which type in Python do you need?

stsouko

Ideally https://pythonhosted.org/bitstring/bitarray.html

stsouko

I want to implement search by Jaccard/tanimoto similarity on bitstrings (popcount(bs1 & bs2)/popcount(bs1 | bs2))

stsouko

moreover. I want to use postgres GIN, but I can't find normal documentation with examples for implementing it

Alexander

@nougmanoff you can define column in the following way: data = Required(str, sql_type='bit(8)') Then in Python you can work with data as with strings: obj.data = '10101010' When working with data in Python, you can manually convert them to bitarray format if you want: array = bitstring.BitArray('0b' + obj.data) I just checked, and it works. Maybe some time in the future we will add specific support for bitstrings in Python.

stsouko

Thank You!

Alexander

Can you tell a bit more about how do you want to use GIN?

stsouko

I found https://github.com/jirutka/smlar/

stsouko

with implemented GIN support. but this module don't support tanimoto metric. also I found https://github.com/eulerto/pg_similarity

stsouko

but this module works only with text

Lucky

Huh, looks interesting

Lucky

Anyone stumbled about a good text search Postgres plugin, e.g. like what elasticsearch offers?

stsouko

I want implement this: http://pubs.acs.org/doi/abs/10.1021/ci200552r

stsouko

and I think GIN is suitable for this

Lucky

I want implement this: http://pubs.acs.org/doi/abs/10.1021/ci200552r

Site is 404

Lucky

Never mind, my wifi derped.

stsouko

for text search http://blog.scoutapp.com/articles/2016/07/12/how-to-make-text-searches-in-postgresql-faster-with-trigram-similarity

Lucky

for text search http://blog.scoutapp.com/articles/2016/07/12/how-to-make-text-searches-in-postgresql-faster-with-trigram-similarity

Thanks!

Lucky

Noticed, on the online editor still strg+A (or cmd+A on mac) isn't working in text fields

Lucky

Also I am missing an option to set sql_default='NOW()'

Lucky

Also the comment is not included in the Postgres database?

Lucky

Also, cascade delete isn't working in the GUI, if I select False, the python code will still contain True

Artur Rakhmatulin

do you press save button ?

Lucky

do you press save button ?

Normally it autosaves

Lucky

And got pony.orm.core.DBSchemaError: Table "StickerPack" cannot be created, because table "stickerpack" (with a different letter case) already exists in the database. Try to delete "stickerpack" table first. with case insensitive Postgres SQL

Lucky

And got pony.orm.core.DBSchemaError: Table "StickerPack" cannot be created, because table "stickerpack" (with a different letter case) already exists in the database. Try to delete "stickerpack" table first. with case insensitive Postgres SQL

Now tracked as https://github.com/ponyorm/pony/issues/222

Lucky

Lol, dat number