Luckydonald
24.10.2016
09:36:23
Ive seen empty repos with a readme for that kind of usecase before.
Alexey
24.10.2016
09:47:11
ok
Thank you for reporing about the bug, Luckydonald!
Luckydonald
24.10.2016
10:25:47
You're welcome :D
Google
Luckydonald
24.10.2016
12:59:59
https://github.com/luckydonald/pbft/blob/953d74f86258603be0a5b06824d92130af9849e8/code/db_proxy/main.py#L22
Alexander
24.10.2016
13:04:20
Wait a sec...
I think the problem is in the fact that Entity.__init__ method was not called
I think you need to add
super(Message, self).__init__()
at the beginning of the Message.__init__ method
Because the Message class is a mixin
I think it is better to not define __init__ in your mixins at all, and don't have any additional attributes which are unknown to Pony. This is because an entity instance can be constructed in several completely different situations. One thing is when you create a new instance, but completely different story is when an instance is retrieved from the database. In that case instance initialization works in a different way, and by overriding __init__ you may broke internal logic
Serge
24.10.2016
13:22:14
AFAK it is recomended way of naming in Python to use Mixin word in such a classes like class MessageMixin:
Luckydonald
24.10.2016
14:02:16
So I made a mixin, I didn't know :D
I just have this InitMessage class, and want to dump it into a Database, but without modifying the original class.
I'll try the super()
The DBInitMessage is subclassing messages.InitMessage and db.Entity, to have InitMessage instances which can be stored in a DB
Updated images above with links to github
I have the Message classes which already do stuff like building itself from json, and converting itself to json, for sending and receiving.
I now want a second program to additionally store them in a database.
To avoid to write all json-related stuff twice, it is just importing the Message classes from the first program.
So my idea was to make a subclass with the pony foo = Required stuff.
Any thoughts how I could realize that?
Oh, now I think I understand.
class A(object):
def __init__(self):
super().__init__()
print("Inside class A init")
class B(object):
def __init__(self):
print("Inside class B init")
class C(A,B):
def __init__(self):
super().__init__()
print("Inside class C init")
So:
>>> C()
Inside class B init
Inside class A init
Inside class C init
Google
Luckydonald
24.10.2016
14:28:48
Alexander
24.10.2016
14:39:44
I think it is more complex than that, if __init__ method expects some arguments
Luckydonald
24.10.2016
14:40:03
class A(object):
def __init__(self, foobar):
print("Inside class A init, foobar={}".format(foobar))
super().__init__()
print("After class A init, foobar={}".format(foobar))
class B(object):
def __init__(self):
print("Inside class B init")
super().__init__()
print("After class B init")
class C(A, B):
def __init__(self):
print("Inside class C init")
super().__init__("hey")
print("After class C init")
So:
In[33]: C()
Inside class C init
Inside class A init, foobar=hey
Inside class B init
After class B init
After class A init, foobar=hey
After class C init
Out[33]:
<__main__.C at 0x1095c1780>
Alexander
24.10.2016
14:47:07
I prefer something like that:
class JsonMixin(object):
def to_json(self):
return {'type': self.type, 'value': self.value}
@classmethod
def from_json(cls, **kwargs):
return cls(**kwargs)
class Message(JsonMixin):
def __init__(self, type, value):
self.type = type
self.value = value
class DBMessage(JsonMixin, db.Entity):
type = Required(int)
value = Required(str)
This way we have two different classes Message and DBMessage. Message is for in-memory data, and DBMessage is for persistent data. The common Json functionality is placed inside separate mixin class which have no __init__ method
Luckydonald
24.10.2016
14:48:54
I see
Other question, how can I model this correctly:
https://editor.ponyorm.com/user/luckydonald/pbft
I get errors like:
pony.orm.core.ERDiagramError: Attribute DBPrevoteMessage.value conflicts with attribute DBInitMessage.value because both entities inherit from DBMessage. To fix this, move attribute definition to base class
But If i move value up to DBMessage that would mean DBProposeMessage has that too.
Alexey
24.10.2016
15:46:53
You need to add the 'value' and 'node' attributes to the DBMessage entity and remove tnem from the descendand classes
Luckydonald
24.10.2016
15:47:16
And set it to optional?
Alexey
24.10.2016
15:47:19
yes
or, create another entity in the middle of the hierarchy and inherit from it the entities that have this attribute
Luckydonald
24.10.2016
15:52:13
but we have
node, leader, value (VoteMessage)
node, leader, (ProposeMessage)
node, value (InitMessage)
So inheration is not really possible, right?
Alexey
24.10.2016
15:55:00
if you make them optional, it would work
Luckydonald
24.10.2016
15:55:17
okey
? 1 new commits to pbft:master:
03dea8e: db proxy Database with mixins? by luckydonald
Now I am puzzled.
I just tried to remove the DBMessage top class, and added the stuff via mixins instead, I just learend about :D
Google
Alexander
24.10.2016
16:08:56
It seems that Entity.__init__ call was still missed
Luckydonald
24.10.2016
16:09:26
But I'm not overriding that any more
Alexander
24.10.2016
16:09:37
You defined __init__ in your mixin and not call base __init__
Luckydonald
24.10.2016
16:12:31
I will rewrite it as one big class.
Alexander
24.10.2016
16:23:59
In my test all works as expected:
from pony.orm import *
class PersonMixin(object):
def hello(self):
print('My name is ' + self.name)
class Person(PersonMixin):
def __init__(self, name):
self.name = name
db = Database('sqlite', ':memory:')
class DBPerson(PersonMixin, db.Entity):
name = Required(str)
sql_debug(True)
db.generate_mapping(create_tables=True)
with db_session:
p1 = Person(name='John')
p1.hello()
p2 = DBPerson(name='Mike')
p2.hello()
with db_session:
p3 = select(p for p in DBPerson).first()
p3.hello()
Luckydonald
24.10.2016
16:39:45
Let me explain my project a bit, maybe that will help understanding my problem.
I have a script (called node) which sends (broadcasts) messages and receives brodcasted messages. Thats basically all it does. Receiving messages and doing a bit of calulation what to send next.
And I am writing another script (called db_proxy) which also reveives the broadcasted messages. This script should decode the stuff from the socket, like the node, but put it in a database.
So I want to use the same code for receiving the Messages.
But in second program i want to dump it into a database, instead of answering with more messages.
The mixin, while perfectly working, is not as readable, because the functionality is split into 2 classes.
I think I will write seperate DBMessage class which has functions to set it's own values according to a given Message instance.
with db_session:
p3 = select(p for p in DBPerson).first()
p = p3.from_db() # eventually calls Person(foo=self.foo, bar=self.bar)
p.foo.do_stuff()
With that I don't have to modify the original classes ( Message or in your example Person)
Because If you take the DB part away, the Mixin classes alone are very confusing.
Alexander
24.10.2016
16:56:40
I think this approach is valid. This way the code may be easier to understand
Luckydonald
25.10.2016
10:17:56
Hey guys
Alexey
25.10.2016
10:18:43
hey
Luckydonald
25.10.2016
10:19:20
SELECT DISTINCT ON (m.type) * FROM (
SELECT * FROM DBMessage ORDER BY date DESC
) as m ORDER BY m.type, m.date DESC
This selects the newest (m.date) DBMessage of each m.type in postgres.
I can't find how to do DISTINCT on pony
Alexey
25.10.2016
10:24:14
Is this what you need? https://docs.ponyorm.com/api_reference.html?highlight=distinct#distinct
Luckydonald
25.10.2016
10:24:45
I couldn't find it as I searched :(
Probalby, yes. Thanks
Alexey
25.10.2016
10:25:23
there is Quick search field here https://docs.ponyorm.com/toc.html
np
Alexander
25.10.2016
10:25:46
I think this is not the same thing. As I understand, "DISTINCT ON" is a special PostgreSQL construction which is not equivalent to DISTINCT
Luckydonald
25.10.2016
10:27:13
yeah, DESTINCT and DESTINCT ON ({column}) are different.
https://www.postgresql.org/docs/9.0/static/sql-select.html#SQL-DISTINCT
Alexander
25.10.2016
10:27:38
You want to select latest message for each message type
Luckydonald
25.10.2016
10:28:35
DESTINCT removes duplicate rows, DESTINCT ON keeps only the first row with the same value of a column
Google
Luckydonald
25.10.2016
10:28:42
Actually I did the wrong thing, here is my correct SQL:
SELECT DISTINCT ON (m.node) * FROM (
SELECT * FROM DBmessage WHERE type = 1
) as m ORDER BY m.node, m.date DESC
But lets stay by the above example :D
Alexander
25.10.2016
10:29:45
Can you use max id instead of a latest date to find the latest message? I think it will be a bit easier to write
Luckydonald
25.10.2016
10:31:22
Alexander
25.10.2016
10:34:43
ok, with dates you can try the following:
select(m for m in DBMessage if (m.type, m.date) in select((m.type, max(m.date)) for m in DBMessage))
But if two messages of the same type may have indentical date (up to microseconds), you will get both messages
Another way is to use raw SQL query:
DBMessage.select_by_sql("""
SELECT DISTINCT ON (m.node) * FROM (
SELECT * FROM DBmessage WHERE type = 1
) as m ORDER BY m.node, m.date DESC
""")
Luckydonald
25.10.2016
10:37:27
Hm, probably a db.select("sql") is better
Btw, how can I apply .show() on the resulting list?
Alexander
25.10.2016
10:40:56
Note that inside raw SQL you can use Python expression, which are converted to query parameters:
message_type = 1
DBMessage.select_by_sql("""
SELECT DISTINCT ON (m.node) * FROM (
SELECT * FROM DBmessage WHERE type = $message_type
) as m ORDER BY m.node, m.date DESC
""")
Luckydonald
25.10.2016
10:41:16
Alexander
25.10.2016
10:42:11
show is the method of QueryResult. select_by_sql returns list of entities, which don't have show method.
Maybe we can add it later
Luckydonald
25.10.2016
10:43:35
Okey, was just to visualize it to myself
How would I do m.date >= NOW() - '1 minute'::INTERVAL?
latest_vote = orm.select(m for m in DBVoteMessage).order_by(orm.desc(DBVoteMessage.date)).first()
I want to only get messages from the last past hour. Best would be to use the server NOW()
Alexander
25.10.2016
17:27:30
You can write
from datetime import datetime, timedelta
select(m for m in DBVoteMessage if m.date >= detetime.now() - timedelta(hours=1))
But Pony will think that expression datetime.now() - timedelta(hours=1) can be evaluated in Python and replaced with a single query parameter.
In order to use server NOW() you can use raw SQL fragment:
select(m for m in DBVoteMessage if m.date >= raw_sql("NOW() - '1 minute'::INTERVAL"))
Luckydonald
25.10.2016
17:32:43
Alexander
25.10.2016
17:33:28
Thanks, we'll fix it
Luckydonald
25.10.2016
17:33:49
But raw_sql is very neat.
Is that executed in the same query?
Google
Alexander
25.10.2016
17:34:07
yes
Luckydonald
25.10.2016
17:34:16
perfect
Alexander
25.10.2016
17:34:42
oops )
Luckydonald
25.10.2016
17:34:48
:D
Alexey
25.10.2016
18:48:19
??
Mikki
26.10.2016
14:05:34
Welcome!
Alexey
26.10.2016
17:18:11
Luckydonald
26.10.2016
20:39:24
You are the 6th Alexander in this group oO
Rozen
26.10.2016
20:39:36
only 6th?