Anonymous
it'sfine.jpg
Lucky
What is the best way to change out the primary key of a database? Probably making another database and export/import? I don't wanna do it manually, would it work to specify two different versions of the database and copy the data in python?
Alexey
primary key of a table?
Anonymous
what's the pony syntax for counting the number of rows in a table?
Lucky
primary key of a table?
https://editor.ponyorm.com/user/luckydonald/Tags The `Sticker` table has a `file_id` column, which is PK. Problem is, Telegram recent changed the way the file IDs are calculated. So I am required to change 'em all. Problem is many tables are referring it. I think the best would to replace it with a Auto increment PK, in case there are future changes.
Anonymous
looks like len(Entity.select()) works, is this a fine way of doing it?
Lucky
looks like len(Entity.select()) works, is this a fine way of doing it?
Wouldn't this load everything into memory? I often do sql `SELECT 1 FROM table` to be more efficient Not sure if needed.
Anonymous
I don't know SQL well enough lol
Alexander
any of this: select(count() for x in MyObject).get() get(count() for x in MyObject) MyObject.select().count() also possible: select(count(x) for x in MyObject).get() get(count(x) for x in MyObject)
Alexander
yes
Alexander
I think this case is too hard for migrations, because change of primary key is a very seldom operation and requires complex cascade actions. I suggest you to have two different files with definition of models, connect to both databases and copy objects from one database to another
Lucky
Yeah, that was what I was thinking, too.
Lucky
I think this case is too hard for migrations, because change of primary key is a very seldom operation and requires complex cascade actions. I suggest you to have two different files with definition of models, connect to both databases and copy objects from one database to another
So I'd be like from fileA import Object as ObjectA from fileB import Objekt as ObjectB page_size = 1000 page_num = 1 while True: with db_session: objects = ObjectA.select(). \ order_by(ObjectA._pk_). \ page(page_num, page_size) For obj in objects: ObjectB(arg1=obj.arg1, arg2=obj.arg2, arg3=obj.arg3, ...) if len(objects) < page_size: break page_num += 1 (typed and copy pasted with phone, lol)
Alexander
yes, but probably you need to copy some dependent objects of other types as well
Alexander
If you want, you can access entity classes as db attributes, sometimes it may be more convenient from models1 import db as db1 from models2 import db as db2 select(x for x in db1.Object)
Alexander
Regarding counting the number of objects, actually the shortest form is: orm.count(x for x in MyObject)
Alexander
len(Entity.select()) form creates all objects in Python, it may be slow and consumes much memory. Other forms just counting the number of rows directly inside the database
Alexander
select count(*) from table1
Lucky
Oh right, forgot the count.
Lucky
select count(*) from table1
But wouldn't SELECT COUNT(1) FROM table1 be better?
Lucky
Because it doesn't need to look at the rows content, but can just count a simple object, here a simple 1?
Alexander
No, it compiles to the same query plan https://asktom.oracle.com/pls/apex/f?p=100:11:0::NO::P11_QUESTION_ID:1156159920245
Alexander
In all databases
Alexander
http://it.blog.adclick.pt/mysql-2/mysql-count1-count-myth/
Lucky
In all databases
You have a postgres reference by hand?
Alexander
https://www.postgresql.org/message-id/11471.1027875769%40sss.pgh.pa.us
Lucky
Huh. The more you know!
Lucky
Thanks!
Alexander
Sure
Lucky
What is the best way to apply triggers automatically in a webserver environment with pony?
Lucky
http://www.revsys.com/blog/2006/aug/04/automatically-updating-a-timestamp-column-in-postgresql/
Alexander
What do you mean 'automatically'?
Lucky
I don't wanna do it manually (ssh'ing into the server etc.) Also, is there already a Pony abstraction for triggers?
Alexander
No, triggers are typically very database-specific. Right now we are creating them manually. Maybe you can write a script which creates a trigger using db.execute(...)
Lucky
Best would be to have a version table, and with that I can store what migrations already were processed?
Alexander
We have such table in our migration tool. I want to finish it, but have no enough time for it, because I'm involved in other projects and they cosume all the time. I really hope that the next week I can concentrate on finishing migration tool
Lucky
Yeah, I know that feeling :/
Lucky
Is it possible to disable $foo automatic resolving, and force it to be needed to manually provide them, like: db.execute("SELECT * FROM Table1 WHERE column1 = $x and column2 = $(a + b)", x="foo", a="some", b="string") ?
Alexander
sql = "SELECT * FROM Table1 WHERE column1 = $x and column2 = $(a + b)" var_dict = {"x": "foo", "a": "some", b: "string"} db.execute(sql, var_dict)
Lucky
Oh great. This also solves the issue with my IDE complaining about unused variables.
Lucky
Something different. I am trying to use obj.to_dict(), but get File "/Users/luckydonald/Documents/Programmieren/Python/DockerTgBot/sticker_tag_bot/code/elastic_pony/data.py", line 8, in default__to_dict return obj.to_dict(obj) File "<string>", line 2, in to_dict File "/Users/luckydonald/Documents/Programmieren/Python/DockerTgBot/virtualenv3.5.1.venv/lib/python3.4/site-packages/pony/utils/utils.py", line 58, in cut_traceback return func(*args, **kwargs) File "/Users/luckydonald/Documents/Programmieren/Python/DockerTgBot/virtualenv3.5.1.venv/lib/python3.4/site-packages/pony/orm/core.py", line 4842, in to_dict attrs = obj.__class__._get_attrs_(only, exclude, with_collections, with_lazy) File "/Users/luckydonald/Documents/Programmieren/Python/DockerTgBot/virtualenv3.5.1.venv/lib/python3.4/site-packages/pony/orm/core.py", line 4073, in _get_attrs_ if only and not isinstance(only, basestring): only = tuple(only) TypeError: 'Testclass' object is not iterable
Alexander
only should be an attribute name or list of atribute names
Lucky
Yeah, it is something deep inside pony.orm.core
Alexander
Instead of obj.to_dict(obj) you need to write just obj.to_dict()
Lucky
Yes. You are right
Lucky
How does Pony handle float/double?
Lucky
Instead of obj.to_dict(obj) you need to write just obj.to_dict()
Indeed, this fixed it. Without further ado, my ponyorm/elasticsearch converter is exisitent. Github link: https://github.com/luckydonald/elastic_pony
Lucky
cc @mshekhter
Lucky
https://github.com/luckydonald/elastic_pony
Lucky
Permanent link to the luckydonald/elastic_pony project you mentioned. (?)
Lucky
Yep, it is working :D
Alexander
How does Pony handle float/double?
Pony represents float/double database type as float type in Python (which actually has double precision). The type of a database column which Pony creates is DOUBLE PRECISION. You can specify sql_type='real' if you want to create a column with a single precision
Lucky
page_size = 1000 page_num = 1 while True: with db_session: objects = MyObject.select(). \ order_by(MyObject.id). \ page(page_num, page_size) process_objects(objects) if len(objects) < page_size: break page_num += 1
Can I store the query until page(page_num, page_size) outside of the while loop? Like this: page_size = 1000 page_num = 1 with orm.db_session: query = table_clazz \ .select() \ .order_by(table_clazz._pk_) if last_modified: query = query.filter(lambda elem: elem.modified > last_modified) # end if while True: objects = query\ .page(page_num, page_size) yield from objects if len(objects) < page_size: break # end if # end while page_num += 1 # end with
Lucky
Now with correct code (order_by(table_clazz._pk_) outside, too)
Alexander
I think you can, if you don't retrieve any objects from db
Lucky
So with yield from objects it does, and I can't do that. Thanks.
stsouko
Hello! Why pony do lazy load for queries like: select((x, y) for x in X for y in Y if x.key1 == y.key2)
stsouko
Is it possible preload all attrs?
Alexander
So with yield from objects it does, and I can't do that. Thanks.
I mean, if you create initial query outside of db_session, and retrieve objects later inside db_session (as you do) it probably should work. But this is not tested
Alexander
Hello! Why pony do lazy load for queries like: select((x, y) for x in X for y in Y if x.key1 == y.key2)
Pony do this because if you retrieve several objects at row like select((x, y) ... then there may be duplicates, and if query contains grouping it may interact badly with some column types like BLOB, etc. Because of this, Pony retrieve only primary keys, and load other attributes in a separate query. In many cases it should be more efficient than to retrieve objects with duplicates in a single query
Alexander
Why do you want to preload all attrs?
stsouko
I have 2 attr in X and 1 in Y except primary key. I use Mixins for convert attr from db to python objects.
Alexander
I don't understand the part about mixins
Anonymous
What is IDE that work flawlessly with pony?
Alexander
In most cases PyCharm works good
stsouko
class X(Ent): x = Required(JSON) def xx(self): return convert(self.x)
stsouko
Or class X(Ent, Mix):
stsouko
class Mix(): def xx...
Lucky
Does pony automatically creates new Tables if I add a db.Entity, and use db.generate_mapping(create_tables=True), even if there already are Tables?
Alexander
Is it possible preload all attrs?
Currently there is no way to load pairs of objects in a single query, but I don't think the additional query hurts performance much in this case, especially if the additional field is of JSON type
Anonymous
does the community edition works well either?
Alexander
Does pony automatically creates new Tables if I add a db.Entity, and use db.generate_mapping(create_tables=True), even if there already are Tables?
Yes, Pony will create table automatically. But if your previous entity references newly added entity, it may be necessary to add column to a previous table