Vitaliy
Thank you too! Pleased to be helpful :)
Жека
sara when we whill have new terminator movie? Will be it like a shit, or it will be nice movie?
Alexander
I think that this time Sara plays on the side of the machines
Жека
it is pity. No chance for salvation. Only one hope remaining - reprogrammed T-800.
Alexander
Hi everybody! We've released Pony ORM 0.7.6
https://blog.ponyorm.com/2018/08/10/pony-orm-release-0-7-6/
https://github.com/ponyorm/pony/releases/tag/0.7.6
Windfarer
👍
Lucky
Interactive mode support for PyCharm console
Tell me more
Alexander
Usually Pony requires that all queries we executed from db_session, and objects cannot be updated after the db_session is over. But this is inconvenient when working from console. Because of this Pony creates implicit db_session when working from console.
Pony has special flag pony.MODE. If it is set to INTERACTIVE, Pony allows writing queries without wrapping them into db_session. To determine the mode Pony performs some special checks for main application module. Checks for interactive mode were:
if not hasattr(main, '__file__'):
pony.MODE = 'INTERACTIVE'
As it turns out, this check is not sufficient when console is opened from PyCharm. So, for PyCharm we've added additional test:
if getattr(main, 'INTERACTIVE_MODE_AVAILABLE', False):
pony.MODE = 'INTERACTIVE'
J J
Oooh
J J
I always wondered why I had to do with db_session in pycharm and not from terminal
J J
Thanks for the update
Alexander
Cause PyCharm console uses some __file__
Alexander
Guys, we have a question.
In one of future Pony releases we will change the way how we name tables. So, we need to upgrade users' databases to change names.
Here is the question: what is the most comfortable way to do it?
We have some ideas, but we are not sure about them.
a) Add db.upgrade(from_pony_version) command. Which will run ALTER TABLE commands.
b) Add special table pony_version which will keep version that was used to create this tables. If you use db.generate_mapping(allow_auto_upgrade=True) Pony will automatically perform ALTER TABLE after connection to database.
c) Like Django require users to manually do ALTER TABLE for new specification.
Any ideas and thoughts will be appreciated.
Alexander
The question is, what way is best suitable to upgrade database on servers. It looks a bit like a migration, but not quite so
Alexander
My ideas why this ways are not that good.
a) User can forgot his previous version.
b) User can have no access to create tables inside production schema
c) Too hard, doesn't seem like pony way.
Alexander
I think (a) is the most convenient way, but I may be wrong
Alexander
So, for example, someone want to upgrade Pony on server. New version of Pony expects new tables names, and also know previous names. Pony connect to database, and need a way to determine do database schema needs upgrade or not, and should Pony perform ALTER TABLE commands right now
Alexander
This is the changes we want to do for table names in some future release:
1) Rename many-to-many tables from entity1_entity2 to entity1_attr1: "course_student" -> "course_students". This is necessary for correct handling of situation when pair of entities has multiple many-to-many relationships
2) If very long name of table or constraint was cut because of database name length limitation, add unique hash to it "some_very_long_table_nam" -> "some_very_long_ta_f0e3df4e". This way each name will be unique
J J
Guys, we have a question.
In one of future Pony releases we will change the way how we name tables. So, we need to upgrade users' databases to change names.
Here is the question: what is the most comfortable way to do it?
We have some ideas, but we are not sure about them.
a) Add db.upgrade(from_pony_version) command. Which will run ALTER TABLE commands.
b) Add special table pony_version which will keep version that was used to create this tables. If you use db.generate_mapping(allow_auto_upgrade=True) Pony will automatically perform ALTER TABLE after connection to database.
c) Like Django require users to manually do ALTER TABLE for new specification.
Any ideas and thoughts will be appreciated.
If I choose A I need to run this script on all instances of my db manually? Test and live? I think I prefer extra table to detect version then
Alexander
So you prefer (b) option
Alexander
with automatic upgrade when new version of pony were installed
J J
I do. Seems simplest
Alexander
Actually (b) looks like a good option to me
Alexander
It has some complicated cases.
What if we see no tables?
What if user named some of these tables by himself?
Again - no access to create new tables.
Alexander
If pony_version table is missing it may mean one of three things:
1) Tables were not created using Pony, Pony just used to connect to pre-existing tables. In that case a developer should not call generate_mapping(create_tables=True) or execute migrate.py apply. It is possible to specify correct _table_ name manually.
2) The database is empty and on generate_mapping(create_tables=True, allow_auto_upgrade=True) call Pony should create all tables, including pony_version table.
3) The database were created using previous version of Pony. It contains most of tables, but some of them have obsolete names. On generate_mapping(create_tables=True, allow_auto_upgrade=True) Pony should upgrade existing tables and add missing ones.
The only problem is how to correctly distinguish (2) and (3) in all situations
Vitaliy
How can I explicitly exit db_session in pycharm console?
Sometimes I need to migrate existing data from previous project via external API. Earlier to do it I called db_session.__exit__() then db.drop_all_tables(with_all_data=True), then db.create_tables(), then db_session.__enter__(). Finally after that I can create new entities with new data, but now it doesn't work, because I never cannot leave db_session.
Alexander
I think you can do
db.commit() # to save previous changes
db.rollback() # to close session
db.drop_all_tables(with_all_data=True)
db.create_tables()
Alexander
But functions explicitly wraped with db_session like (db.drop_all_tables and db.create_tables) should auto-close previous implicit db_session. I need to check that
Vitaliy
Fantastic! It works! :)
Alexander
I tried, and it works from PyCharm console:
>>> from pony.orm.examples.university1 import *
>>> populate_database()
>>> select(s for s in Student).show()
>>> db.drop_all_tables(with_all_data=True)
>>> db.create_tables()
So, it seems, nothing special is required, and implicit db_session auto-closes correctly before db.drop_all_tables
РИТ-Сервис
Добрый день
Подскажите как реализоватьт Concrete Table Inheritance. Только ручками? Прописывая в каждом классе одинаковые атрибуты.
Например, есть класс Документ. У каждого документа есть Номер и Дата и возможно другие общие поля. И видов документов может быть много.
Может разработать разные виды наследования?
Alexander
Good day Alexander! This chat is in English.
At this moment Pony support single-table inheritance only, we plan to add other types of inheritance in the future.
If you want to have several classes with similar attributes, it is possible to do, but I'm away from the computer and will show it later
РИТ-Сервис
Alexander
It is possible to use similar approach as shown in this post
Alexander
In principle, it is possible to create entity classes dynamically. Simplified example:
from pony.orm import *
from pony.orm import core
db = Database()
def define_person_entity(db):
attrs = []
attrs.append(('name', Required(str)))
attrs.append(('age', Required(int)))
Person = core.EntityMeta('Person', (db.Entity,), dict(attrs))
define_person_entity(db)
db.bind('sqlite', ':memory:')
sql_debug(True)
db.generate_mapping(create_tables=True)
with db_session:
p1 = db.Person(name='John', age=18)
p2 = db.Person(name='Mike', age=20)
persons = select(p for p in db.Person)[:]
You can use the same idea to generate entity definitions based on information from some configuration file
Vitaliy
Hi, Alexander! Is there a way to share pony objects across different threads?
Alexander
Hi Vitaliy! Can you describe your use-case in more details?
Vitaliy
In my project there are support ticket system. I working now on notifications when a new Message has been added. In after_insert callback I call async task (function in new thread) for sending an e-mail to admin. Currently I pass self argument from Message.after_insert to send_notification task.
Alexander
In a new thread you should use a separate db_session. You can pass self.id instead of self and re-fetch object by id, or you can pickle some data (including Pony objects) in the first db_session and unpickle them in the second db_session
Alexander
If the function that you wrap with db_session is generator, the db_session will suspend on yield and resume on next(gen). It is possible to resume generator in another thread. It was implemented for better compatibility with async frameworks like Tornado. Maybe you can use it, but for your case it looks like an overkill
Vitaliy
I use flask and have subclassed Thread to invoke tasks within app and request contexts. It seems pickling can help, thanks
Matthew
data = Required(LongUnicode, 1024 * 1024 * 16)
Matthew
TypeError: Max length is not supported for CLOBs
Matthew
What am I doing wrong?
Matthew
I'm using postgres 10.5
Matthew
Fixed it :)
Matthew
no length arg
Alexander
yes )
Matthew
Does Pony have any length limits, or is it whatever the database supports?
Alexander
For CLOBs? No, only database provider restrictions
Matthew
great :)
Windfarer
Hi Alexander, is there any way to perform batch insert or update? And especially in raw SQL? thanks.
Carel
Hi, I’m playing around with pony and tornado and it’s gone really nicely so far, pony is really quite awesome. I’ve hit a snafoo though, I can’t seem to perform “Entity.to_json()” conversions as I haven’t setup user access control the pony docs don’t mention this scenario. I was wondering if there was an example or doc that I could read up on. Most of my searches end up with info about actual tornadoes.
stsouko
use to_dict instead
Matthew
Yep, then you have a normal dict which tornado can handle how it wishes
Matthew
Is there a way I can make pony show the connection string it is using to connect to postgres?
Matthew
some internal private value it is fine, it is only for debugging something else
Carel
Right, thanks, I'll switch to dict then, for reference I did manage to find these pages https://docs.ponyorm.com/ponyjs_security.html and https://stackoverflow.com/q/41030262. I also tried importing the Pony.flask module in the hopes that it might discuss how to get this working but I get "Module not found" instead, presumably this is deprecated ?
Matthew
ponyjs is javascript frontend stuff
Matthew
sorry I think I'm wrong
Carel
Ok, do you know if there is more documentation for the PonyJS stuff ?
Matthew
I don't know I'm afraid. I'd suggest controlling access in your web framework
Carel
For the connection infor have you tried enabling the database debug switch, it shows all the connections/disconnections to SQLite, perhaps it does the same for Postgres (I haven't tried with a real db yet) "set_sql_debug(True)"
Carel
Thanks Mr. Bell, I'll code dig this afternoon then :D and use dicts for the interim.
Matthew
cool. I tried sql_debug but unfortunately it didn't work
Carel
Can you wireshark it ?
Matthew
Maybe
Matthew
Does pony connect to postgres via the unix socket by default?
Carel
Have you installed psycopg2 ?
Matthew
I solved the problem
Matthew
Python was working fine, because it was using unix socket
Matthew
other program was connecting over TCP so needed a password
Alexander
Hi Alexander, is there any way to perform batch insert or update? And especially in raw SQL? thanks.
Hi Windfarer, at this moment Pony does not provide batch inserts or updates, you can get native DBAPI connection using db.get_connection() and manually perform
cursor = connection.cursor()
cursor.executemany(custom_sql, list_of_args)
In PostgreSQL you need to perform psycopg2.extras.execute_batch / execute_values instead of executemany in order to get good performance
Alexander
Hi Carel, PonyJS remains unreleased. We plan to rewrite PonyJS but at this moment have no resources for that
Alexander
At this moment you need to use to_dict, I hope later we can provide something better
Carel
At this moment you need to use to_dict, I hope later we can provide something better
Ok, so I’ll skip to_json for now. Thank you for your response. It’s a really nice api by the way. Also is the PonyJS mixed in with the Python code at the moment ? Or is it hosted else where ? I’ll pull the repo in a moment and have a poke around.
J J
For getting json representation from my models I have used also the Marshmallow library. Maybe worth looking into