Hi guys, i just started using ponyorm through my telegram bot

I have no idea how to store date, I always get error, I think my date format from python was rigth

Anyone know to deal with this ? Sorry for silly question

Can you give an example of how you do it?

that is my entity

that is the function that I use to passing the data

And what is the error?

the error come from the framework (telegram-bot not pony), but from what i learned before, it caused by wrong datatype input

I want to pass yyyy-mm-dd format to date type, is it possible ?

If score is an instance of Score, then score.taken would be instance of date. If you want to get a date in format of yyyy-mm-dd, you can use isoformat method of date object: score.taken.isoformat()

Thanks sir, I'll try it

If score is an instance of Score, then score.taken would be instance of date. If you want to get a date in format of yyyy-mm-dd, you can use isoformat method of date object: score.taken.isoformat()

Sir, I found the actual problem, the "max" option on my Score entity doesn't work. I checked on my mysql table, I get int(11) not int(2) as I expected

max option allows to check value assigned to the attribute in Python. If you specify that max=2 that means that the value may be 0, 1, 2, -1, -100, etc., but no greater then 2

int(11) type means that the column type is a standard 4-byte int value

max option allows to check value assigned to the attribute in Python. If you specify that max=2 that means that the value may be 0, 1, 2, -1, -100, etc., but no greater then 2

Oh I see, but how to limit int into specific length ?

you can set size option. It may be 8, 16, 32, 64

If i what to limit my int into 2 digits only, like the varchar, is it posible to use size option ?

*want

you can specify max=99. It is also possible to specify size=16 if the goal is to reduce database size

you can specify max=99. It is also possible to specify size=16 if the goal is to reduce database size

Thank you !👍

Hi! I need to divide my query results by chunks. Is there a conventional way to do this?

I've tried to treat query results like a usual iterable, but it is working unexpectedly

if i use something like: def chunks(iterable, n): it = iterable while True: chunk = tuple(itertools.islice(it, n)) if not chunk: return yield chunk with for-loop like this: for chunk in chunks(it, 300): for l in chunk: it is working forever

and if i use function like this: def chunkify(it, n): for i in range(0, len(it), n): yield it[i:i + n] with the same for-loop it works n times no matter how many records there was

Oh, I forgot to say that i use query with a collection, like this: reviews = select((l.event_id, l.action, s.id, s.geometry) for l in Logged_actions for s in Spot)

Pony fetches all selected rows at once. For typical web application a query result should not contains too many rows. If you want to fetch millions of rows and process them in a for-loop, that looks like an antipattern to me. If you really need to process very big amount of rows, you can fetch each chunk of rows with a separate query: last_id = 0 page_size = 100 while True: with db_session: objects = select(x for x in MyEntity if x.id > last_id).order_by(x.id)[:page_size] if not objects: break for x in objects: do_something_with(x) last_id = chunk[-1].id Note that I process each chunk of objects in a separate db_session. If I use the same session, then previously retrieved objects will continue sit in it until the end of db_session. In the future we can add a possibility to lazily fetch objects. But this is non trivial, because we need to garbage collect previously loaded objects, and this is not easy to do without exiting from db_session

I'm trying to translate data from DB to elasticsearch index, thus it is not a web application and there can be a lot of rows.

Functions that I wrote previously work pretty well with usual query (select(x for x on x)), but do not work for a query with collection ((x, y) for x in x for y in y)

https://www.postgresql.org/docs/9.2/static/plpgsql-cursors.html

I think Pony has no support this, I just remember this feature in postgresql

PonyORM does not support PostgreSQL server-side cursors (yet?). I think even if you query select pairs of objects select((x,y) for x in X for y in Y) You can add sensible order_by expression and split objects into chunks as in my example

I think partial retrieve is better, cause explicit.

Like Alexander say before 👍

My goal was to reduce time and resources with removing extra queries. I dont see this as a good solution, if, instead of 1 query I perform 20 or 200. I will try adding .order_by() clause, but maybe you try and fix this?

I think it may be better to perform multiple big queries then one super-big query. But returning to the initial question: > I need to divide my query results by chunks. Why do you need this? My initial understanding was that a single query was bad for performance and you want to optimze it. Was my understanding correct?

I need this because it is easier to feed chunks to elastic.

It is actually not so critical, but would be better if query results behaved similar.

@mshekhter Is that piece if code you write open source? Because transferring data in an elastic search index seems to be something I need to do too, to improve performance

So maybe I could profit xD

hello

How to specify a schema when using Oracle DB. My user hasn't a default schema

thx

How to specify a schema when using Oracle DB. My user hasn't a default schema

I found editor.ponyorm.com very helpful

But you'll need a desktop browser , can't use that mobile

lol ) sorry for my english ) i mean SCHEMA for USER in OracleDB

I have _test_ user in DB. For use _select_ queries i must write like _Select * From SCHEMA.TABLE_

Use ` to mark code in telegram: `test`

Can someone explain the StopIterator change? I'm getting a DeprecationWarning but I dont' know what to change it too

I'm on 3.6

Huh, not that I can help, but what is your code?

for url in post.media: if found >= count: raise StopIteration

bears.py:65: DeprecationWarning: generator 'get_timeline_media' raised StopIteration

How to specify a schema when using Oracle DB. My user hasn't a default schema

You can specify schema name for each entity: class MyEntity(db.Entity): _table_ = ['myschema', 'mytable'] ...

Beware, that in Oracle there is a restriction on an object name length - it is 30 char max. When pony create indexes and foreign keys, the default name of a foreign key looks like FK_TABLENAME__COLUMNNAME, and if the name exceed the limit of 30 characters it will be cut. It can lead to error where different constraints have the same name (after cut). It will be fixed in the next release.

Thanks for the detailed answer

@luckydonald actually not, but im thinking about making open-source module based on it

so, if u're interested, we can work together

Uh, not bad

hey all, I'm having trouble with inserting a Decimal type into sqlite3. I know that sqlite3 doesn't have native Decimal types, rather it has a Numeric type...but Pony creates the schema as Decimal... Is the canonical "fix" to create the schema as str and provide sqlite with an adapter?

so, if u're interested, we can work together

Using elastic dsl?

hey all, I'm having trouble with inserting a Decimal type into sqlite3. I know that sqlite3 doesn't have native Decimal types, rather it has a Numeric type...but Pony creates the schema as Decimal... Is the canonical "fix" to create the schema as str and provide sqlite with an adapter?

Can you specify the details of the problem?

Hi everybody!

Tried to perform some update query via db.execute and got this: pony.orm.dbapiprovider.ProgrammingError: can't adapt type 'builtin_function_or_method'

what is the reason of this error and how do i fix it?

Maybe you specify date module instead of date.date class or something like that?

i've got from datetime import date

Can you show the traceback?

Traceback (most recent call last): File "/usr/local/lib/python3.5/site-packages/pony/orm/dbapiprovider.py", line 48, in wrap_dbapi_exceptions try: return func(provider, *args, **kwargs) File "/usr/local/lib/python3.5/site-packages/pony/orm/dbproviders/postgres.py", line 216, in execute else: cursor.execute(sql, arguments) psycopg2.ProgrammingError: can't adapt type 'builtin_function_or_method' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/mark/deep-fish-map/src/indexer.py", line 408, in <module> main_loop() File "/Users/mark/deep-fish-map/src/indexer.py", line 398, in main_loop spind.clarify() File "/Users/mark/deep-fish-map/src/indexer.py", line 123, in clarify self._make_clarification(id) File "<string>", line 2, in _make_clarification File "/usr/local/lib/python3.5/site-packages/pony/orm/core.py", line 413, in new_func try: return func(*args, **kwargs) File "/Users/mark/deep-fish-map/src/indexer.py", line 152, in _make_clarification db.execute(sql) File "<string>", line 2, in execute File "/usr/local/lib/python3.5/site-packages/pony/utils/utils.py", line 58, in cut_traceback return func(*args, **kwargs) File "/usr/local/lib/python3.5/site-packages/pony/orm/core.py", line 628, in execute return database._exec_raw_sql(sql, globals, locals, frame_depth=3, start_transaction=True) File "/usr/local/lib/python3.5/site-packages/pony/orm/core.py", line 640, in _exec_raw_sql return database._exec_sql(adapted_sql, arguments, False, start_transaction) File "/usr/local/lib/python3.5/site-packages/pony/orm/core.py", line 703, in _exec_sql connection = cache.reconnect(e) File "/usr/local/lib/python3.5/site-packages/pony/orm/core.py", line 1511, in reconnect if not provider.should_reconnect(exc): reraise(*sys.exc_info()) File "/usr/local/lib/python3.5/site-packages/pony/utils/utils.py", line 85, in reraise try: raise exc.with_traceback(tb) File "/usr/local/lib/python3.5/site-packages/pony/orm/core.py", line 701, in _exec_sql try: new_id = provider.execute(cursor, sql, arguments, returning_id) File "<string>", line 2, in execute File "/usr/local/lib/python3.5/site-packages/pony/orm/dbapiprovider.py", line 50, in wrap_dbapi_exceptions except dbapi_module.ProgrammingError as e: raise ProgrammingError(e) pony.orm.dbapiprovider.ProgrammingError: can't adapt type 'builtin_function_or_method'

In your _make_clarification method you execute some raw SQL query in line 152: db.execute(sql) In that query you use some parameter. It seems that you use some function call to evaluate param value and forgot parentheses after the function call. Something like: x = date.today sql = 'select a from Table1 where b < $x' It should be instead: x = date.today() sql = 'select a from Table1 where b < $x'

хм

ой

thank you!

Sure

Has there been any work done on something like Flask-Admin for PonyORM

Has there been any work done on something like Flask-Admin for PonyORM

No, nothing like that yet. We have plans to add it to the https://editor.ponyorm.com

No, nothing like that yet. We have plans to add it to the https://editor.ponyorm.com

That doesn't sound like a good idea.

That doesn't sound like a good idea.

Here is why. - Flask-Admin is open source The editor isn't. I can't validate what it does with my database. - Flask-Admin is something you have on your own server. This means you don't have to expose your database to the public to make the external web service https://editor.ponyorm.com work. Also that means someone evil at ponyorm could change data in your database If they are already connected. Not saying you guys are evil, but it introduces an unnecessary security risk.