It should work for simple mugrations in PostgreSQL. The orm-migration branch is based on current release, currently I'm in process of merging it with the code of upcoming release. Current migration code should handle simple migrations like adding a new attribute, but not complex migrations like changing inheritance

ok thank you

You are welcome

Hello, is there a common usage on naming module with entity definition like django models ?

Yes, it is common with Pony to make a module models.py which looks something like that: from pony import orm db = orm.Database() class Foo(db.Entity): ... class Bar(db.Entity): ... And then import it from main.py: from models import db import settings db.bind(*settings.db_params) db.generate_mapping(create_tables=True)

OK good thank you

Is there a suggested pattern for splitting models in a project into sub modules?

It isn't obvious me how to have references to other modules when doing that

Something like this: # base_models.py from pony import orm db = orm.Database() class Foo(db.Entity): bars = Set('Bar') # or Set(lambda: db.Bar) # another_models.py from pony import orm from base_models import db class Bar(db.Entity): foos = Set('Foo') # or Set(lambda: db.Foo) # all_models.py from base_models import db, Foo from another_models import Bar class Baz(db.Entity): name = Required(str) # main.py from pony import orm from all_models import db, Foo, Bar, Baz import settings db.bind(**settings.db_params) db.generate_mapping(create_tables=True) select(bar for bar in db.Bar if ...) Note that you can access entities as attributes of db object, somethimes it makes things easier

Also, it may be convenient to define entities inside a function: # models.py from pony import orm import models1, models2 db = Database() models1.define_entities(db) models2.define_entities(db) # models1.py def define_entities(db): class Foo(db.Entity): bars = Set('Bar') # or Set(lambda: db.Bar) # models2.py def define_entities(db): class Bar(db.Entity): foos = Set('Foo') # or Set(lambda: db.Foo) # main.py import settings from models import db db.bind(**settings.db_params) db.generate_mapping(create_tables = True) select(bar for bar in db.Bar if ...)

Ah I didn't realise about entities being on the db object, cool :)

Also, it may be convenient to define entities inside a function: # models.py from pony import orm import models1, models2 db = Database() models1.define_entities(db) models2.define_entities(db) # models1.py def define_entities(db): class Foo(db.Entity): bars = Set('Bar') # or Set(lambda: db.Bar) # models2.py def define_entities(db): class Bar(db.Entity): foos = Set('Foo') # or Set(lambda: db.Foo) # main.py import settings from models import db db.bind(**settings.db_params) db.generate_mapping(create_tables = True) select(bar for bar in db.Bar if ...)

Functions have the main disadvantage that IDEs like Pycharm can't handle it, and you won't get code autocomplete

There should be a pattern like flasks Blueprints

A register-me-later approach

Like making the models subclass of Database.Entity, and having a db.register(Foo) or a Foo.register(db)

I need to think about it...

Maybe that wouldn't need to replace the current way, but let the static version work without a DB at first.

A question about tests.How do you deal tests with a test database in pony. Is there a way to mock db or Database.bind or anything like that ? I'd like to do a pytet plugin for pony.

A question about tests.How do you deal tests with a test database in pony. Is there a way to mock db or Database.bind or anything like that ? I'd like to do a pytet plugin for pony.

A sqlite db can be in memory

yes of course but because entity refers to db, how to change, for example the db provider ?

if unittest: db = else: db =

Maybe?

no because database definition occurs with "bind" call. The thing is i'd like to change change the bind command at test setup to point to another db

I think it is possible to structure code like this: # settings.py db_params = dict( production=dict(provider='postgres', host='localhost', ...) test=dict(provider='sqlite', filename='testdb.sqlite', ...) ) # main.py from my_flask_app import app from models import db import settings if app.config['TESTING']: db_params = setting.db_params['test'] else: db_params = settings.db_params['production'] db.bind(**db_params)

I set a different environment variable for testing.

I do too. I use pytest-env to check if code is runned via pytest or not.s o for my pytest, plugin I will specify that test settings are managed directly by user not by plugin.

So you can bind different parameters depending on environment variable.

ok, here is a first try for a pytest plugin : https://github.com/jgirardet/pytest-ponyorm . install via pip. Let me know your opinions

What is that?

Hello, my name is Artemii (IU Student). I really could not find information about it. How I can modify structure of table (in my case I just want to add on more field) without recreating all tables. I need it because now I have already several thosounds records and I cannot jast delete tables and creating again. Thanks in advance.

What database do you use?

MySQL

So you can use ALTER_TABLE command Or You can use orm-migrations, it is a branch on github

Okey, I catch it, can give me url to this branch?

https://github.com/ponyorm/pony/tree/orm-migrations

Okey, thank you)

The migration tool is in beta state, it should work for simple changes like adding a new column. I suggest you to make a backup before using migrations

Here is the description of how to use migrations: https://github.com/ponyorm/pony/tree/orm-migrations/pony/migrate

I am not understand how to install version of Pony ORM with migrations support, in the case now I use Pony ORM 0.7.3 (installed using pip)

You can pip uninstall pony and then clone github repository https://github.com/ponyorm/pony and then checkout branch orm-migrations. You also need to add pony outer directory to PYTHONPATH

shouldn't `pip install git+https://github.com/ponyorm/pony.git@orm-migrations` also work? that's how I'm installing a future branch of a different package

Yes, I think it is possible too

And should be easier

shouldn't `pip install git+https://github.com/ponyorm/pony.git@orm-migrations` also work? that's how I'm installing a future branch of a different package

Okey, i uninstalled pony, install future branch. Imports in python should be the same or how will it work?

It is very important to make a backup. Dont forget it. Imports are the same.

Here is the description of how to use migrations: https://github.com/ponyorm/pony/tree/orm-migrations/pony/migrate

Read doc how to use migrations at the link

Yes, already done it in the beginning, will explore it

You can pip uninstall pony and then clone github repository https://github.com/ponyorm/pony and then checkout branch orm-migrations. You also need to add pony outer directory to PYTHONPATH

Better to install the git version with pip

shouldn't `pip install git+https://github.com/ponyorm/pony.git@orm-migrations` also work? that's how I'm installing a future branch of a different package

Yeah, that is probably better as it uses pip, and you can even add a explicit commit to the requirements.txt

Добрый день. Подскажите подалуйста, почему я не могу получить из базы CLOB? Python 2.7 Required(LongStr) exc_type = <type 'exceptions.ValueError'> args = ('Value for attribute CDR_CREDIT_ORDER_T.ORDERDATA cannot be converted to unicode: <cx_Oracle.LOB object at 0x0BE4BDE8>',) kwargs = {} def throw(exc_type, *args, **kwargs): if isinstance(exc_type, Exception): assert not args and not kwargs exc = exc_type else: exc = exc_type(*args, **kwargs) exc.__cause__ = None try: if not (pony.MODE == 'INTERACTIVE' and options.CUT_TRACEBACK): > raise exc E ValueError: Value for attribute CDR_CREDIT_ORDER_T.ORDERDATA cannot be converted to unicode: <cx_Oracle.LOB object at 0x0BE4BDE8>

Hi. Can you provide a more detailed tractback? Добрый день. Можете прислать более полный стэктрейс, чтобы понять в какой строке возникает ошибка?

================================== FAILURES =================================== _________ Test_cash_order_send.test_order_status_in_cd[cash-5343138] __________ self = <tests.dbo2.aplication_service.test_application_service.Test_cash_order_send instance at 0x0C059DC8> cdr_orderdata = <function issss at 0x0C05E070>, id = '5343138', data = 'cash' @db_session @pytest.mark.parametrize( 'data, id', [ ('cash', Data.id) ] ) #@pytest.mark.run(order=2) def test_order_status_in_cd(self, cdr_orderdata, id, data): > a = cdr_orderdata(integration=id, xml='C:\Framework\Autotests\\tests\dbo2\\aplication_service\cash.xml', type=data) tests\dbo2\aplication_service\test_application_service.py:47: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ tests\dbo2\aplication_service\conftest.py:55: in issss cdr_order = slct(CDR_CREDIT_ORDER_T, 'ORDERID', id).ORDERDATA <auto generated wrapper of __get__() function>:2: in __get__ ??? C:\tools\Python27\lib\site-packages\pony\utils\utils.py:58: in cut_traceback return func(*args, **kwargs) C:\tools\Python27\lib\site-packages\pony\orm\core.py:2099: in __get__ value = attr.get(obj) C:\tools\Python27\lib\site-packages\pony\orm\core.py:2109: in get val = vals[attr] if attr in vals else attr.load(obj) C:\tools\Python27\lib\site-packages\pony\orm\core.py:2091: in load dbval = attr.parse_value(row, offsets) C:\tools\Python27\lib\site-packages\pony\orm\core.py:2054: in parse_value val = attr.validate(row[offset], None, attr.entity, from_db=True) C:\tools\Python27\lib\site-packages\pony\orm\core.py:2352: in validate val = Attribute.validate(attr, val, obj, entity, from_db) C:\tools\Python27\lib\site-packages\pony\orm\core.py:2030: in validate % (attr, unicode.__name__, truncate_repr(val))) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ exc_type = <type 'exceptions.ValueError'> args = ('Value for attribute CDR_CREDIT_ORDER_T.ORDERDATA cannot be converted to unicode: <cx_Oracle.LOB object at 0x0BE4BDE8>',) kwargs = {} def throw(exc_type, *args, **kwargs): if isinstance(exc_type, Exception): assert not args and not kwargs exc = exc_type else: exc = exc_type(*args, **kwargs) exc.__cause__ = None try: if not (pony.MODE == 'INTERACTIVE' and options.CUT_TRACEBACK): > raise exc E ValueError: Value for attribute CDR_CREDIT_ORDER_T.ORDERDATA cannot be converted to unicode: <cx_Oracle.LOB object at 0x0BE4BDE8>

C:\tools\Python27\lib\site-packages\pony\utils\utils.py:98: ValueError ---------------------------- Captured stdout call ----------------------------- 2018-03-27 15:12:15,640 - pony.orm - INFO - GET CONNECTION 2018-03-27 15:12:15,641 - pony.orm.sql - INFO - INSERT INTO "CDR_CREDIT_ORDER_T" ("ORDERID", "INTEGRATIONID", "ORDERDATA") VALUES (:p1, :p2, :p3) 2018-03-27 15:12:15,677 - pony.orm.sql - INFO - SELECT "c"."ORDERID", "c"."INTEGRATIONID", "c"."PHONENUMBER", "c"."LASTNAME", "c"."FIRSTNAME", "c"."MIDDLENAME", "c"."STATUS", "c"."CREATEDT" FROM "CDR_CREDIT_ORDER_T" "c" WHERE "c"."ORDERID" = :p1 2018-03-27 15:12:15,686 - pony.orm.sql - INFO - SELECT "ORDERDATA" FROM "CDR_CREDIT_ORDER_T" WHERE "INTEGRATIONID" = :p1 2018-03-27 15:12:15,730 - pony.orm - INFO - ROLLBACK 2018-03-27 15:12:15,737 - pony.orm - INFO - RELEASE CONNECTION ------------------------------ Captured log call ------------------------------ core.py 103 INFO GET CONNECTION core.py 113 INFO INSERT INTO "CDR_CREDIT_ORDER_T" ("ORDERID", "INTEGRATIONID", "ORDERDATA") VALUES (:p1, :p2, :p3) core.py 113 INFO SELECT "c"."ORDERID", "c"."INTEGRATIONID", "c"."PHONENUMBER", "c"."LASTNAME", "c"."FIRSTNAME", "c"."MIDDLENAME", "c"."STATUS", "c"."CREATEDT" FROM "CDR_CREDIT_ORDER_T" "c" WHERE "c"."ORDERID" = :p1 core.py 113 INFO SELECT "ORDERDATA" FROM "CDR_CREDIT_ORDER_T" WHERE "INTEGRATIONID" = :p1 core.py 103 INFO ROLLBACK core.py 103 INFO RELEASE CONNECTION ============================== warnings summary =============================== test_application_service.py::Test_cash_order_send::()::test_order_status_in_cd[cash-5343138] C:\tools\Python27\lib\site-packages\pony\orm\core.py:2360: DatabaseContainsIncorrectEmptyValue: Database contains NULL for required attribute CDR_CREDIT_ORDER_T.PHONENUMBER DatabaseContainsIncorrectEmptyValue) C:\tools\Python27\lib\site-packages\pony\orm\core.py:2360: DatabaseContainsIncorrectEmptyValue: Database contains NULL for required attribute CDR_CREDIT_ORDER_T.LASTNAME DatabaseContainsIncorrectEmptyValue) C:\tools\Python27\lib\site-packages\pony\orm\core.py:2360: DatabaseContainsIncorrectEmptyValue: Database contains NULL for required attribute CDR_CREDIT_ORDER_T.FIRSTNAME DatabaseContainsIncorrectEmptyValue) C:\tools\Python27\lib\site-packages\pony\orm\core.py:2360: DatabaseContainsIncorrectEmptyValue: Database contains NULL for required attribute CDR_CREDIT_ORDER_T.MIDDLENAME DatabaseContainsIncorrectEmptyValue) C:\tools\Python27\lib\site-packages\pony\orm\core.py:2360: DatabaseContainsIncorrectEmptyValue: Database contains NULL for required attribute CDR_CREDIT_ORDER_T.STATUS DatabaseContainsIncorrectEmptyValue) -- Docs: http://doc.pytest.org/en/latest/warnings.html ==================== 1 failed, 5 warnings in 8.51 seconds ===================== Process finished with exit code 0

сам селект выглядит так @db_session def slct(table, column, id): return select(c for c in table if getattr(c, column) == id)[:][0] slct(CDR_CREDIT_ORDER_T, 'ORDERID', id).ORDERDATA

Can you read raw value of attribute and check its value? It seems that that value cannot be converted to unicode: def check_value(id) value = db.select(""" select ORDERDATA from CDR_CREDIT_ORDER_T where ORDERID = $id """)[0] print(repr(value)) if hasattr(value, 'read'): value = value.read() print(repr(value)) value = unicode(value) print(value)

I get this error pony.orm.dbapiprovider.DatabaseError: ORA-00936

this strangely

"The ORA-00936 happens most frequently: 1 - When you forget list of the column names in your SELECT statement. 2. When you omit the FROM clause of the SQL statement. "

can you check generated SQL?

Yes, I did it. in the DBeaver, this query works and returns CLOB

When you executing a query, do you specify 5 as a literal value embedded inside a query or as a parameter (like $id, as I showed)?

I did so @db_session def check_value(): value = cdr.select(""" select ORDERDATA from CDR_CREDIT_ORDER_T where INTEGRATIONID = '5343138' """)[0] print(repr(value)) if hasattr(value, 'read'): value = value.read() print(repr(value)) value = unicode(value) print(value)

try to specify id value outside of query: @db_session def check_value(): id = '5343138' value = cdr.select(""" select ORDERDATA from CDR_CREDIT_ORDER_T where INTEGRATIONID = $id """)[0] print(repr(value)) if hasattr(value, 'read'): value = value.read() print(repr(value)) value = unicode(value) print(value)

I got it again ORA-00936

what cx_oracle version do u use?

какая версия cx_oracle? может проблема в этом

6.0rc1

try cx_oracle 5.3 with instantclient11.2

u can use docker image if u want https://hub.docker.com/r/binarycat/cx_oracle/

Also try to use native DBAPI cursor: @db_session def check_value(): connection = cdr.get_connection() cursor = connection.cursor() cursor.execute(""" select ORDERDATA from CDR_CREDIT_ORDER_T where INTEGRATIONID = '5343138' """) value = cursor.fetchone()[0] print(repr(value)) if hasattr(value, 'read'): value = value.read() print(repr(value)) value = unicode(value) print(value)

short how_to for docker image https://github.com/catbinary/cx_oracle_example

Also try to use native DBAPI cursor: @db_session def check_value(): connection = cdr.get_connection() cursor = connection.cursor() cursor.execute(""" select ORDERDATA from CDR_CREDIT_ORDER_T where INTEGRATIONID = '5343138' """) value = cursor.fetchone()[0] print(repr(value)) if hasattr(value, 'read'): value = value.read() print(repr(value)) value = unicode(value) print(value)

thanks it helped) I got what lies in CLOB

cool

so, the function worked till the end without any exception?

yes

That's strange

very strange

I'm preparing the test data. I use fixtures. there may be a problem in this

can be a fixture removes the data

thanks for the help @akozlovsky @binarycat0

Maybe there is some bug in Oracle provider, but at this moment I cannot reproduce the situation. If the problem will remain, we can debug it together in personal chat to discover the root of the problem

cx_oracle 6.x release not good... i try use it, but return to 5.x version

that's interesing

probably the problem is this

I thought that in the xml

but there is an xml shielded string

cx_oracle 6.x release not good... i try use it, but return to 5.x version

I try cx_oracle 5.3

Hello, I'm tryning to have some fixture usable in shell. I get a TransactionError. Here is the Code :