Wednesday, 28 December 2011< ^ >
arjan has set the subject to: Zotonic - the Erlang CMS
Room Configuration

[00:13:23] maas.maarten.zeeman leaves the room
[07:52:18] vio42 joins the room
[07:52:23] vio42 leaves the room
[08:07:10] Thomas Spellman joins the room
[08:07:44] <Thomas Spellman> hello?
[09:58:00] arjan joins the room
[10:09:04] vio42 joins the room
[10:09:10] vio42 leaves the room
[10:45:07] <arjan> hi Thomas
[10:55:05] vio42 joins the room
[10:55:21] vio42 leaves the room
[10:58:04] vio42 joins the room
[10:58:06] vio42 leaves the room
[11:14:54] <wok> Reworking the handoff algorithms to fit our services model.
[11:15:45] Maas-Maarten Zeeman joins the room
[11:16:06] <Maas-Maarten Zeeman> hi
[11:16:26] <wok> Anybody any good ideas/pointers to implement a simple back-pressure in Erlang?
[11:17:09] <Maas-Maarten Zeeman> don't even know what is
[11:17:13] <Maas-Maarten Zeeman> :-/
[11:17:15] <wok> Want to prevent DOS attacks with handoff data when a node comes back alive :-) Am now using throttling on the sender's side, guess the recipient should have some say in how much he can handle.
[11:17:27] <wok> back pressure -> recipient tells sender to slow down
[11:18:13] <Maas-Maarten Zeeman> k
[11:18:30] <wok> Maybe the recipient can just send some 'minimum delay to wait', might work well in practice.
[11:18:48] <arjan> yes
[11:18:58] <arjan> or some speed indication, in reqs/time
[11:19:11] <Maas-Maarten Zeeman> yes
[11:19:16] <wok> or let the sender see how busy the recipient is?
[11:19:39] <Maas-Maarten Zeeman> could depend on hw i guess
[11:20:05] <wok> yes, and how fast updates are handled
[11:20:43] <wok> so, decision is to let the recipient issue back pressure or let the sender decide that some slow down is appropriate
[11:21:09] <wok> Could also let the recipient kill some requests and answer with a minimum delay
[11:21:14] <Maas-Maarten Zeeman> can handoffs be done synchronized?
[11:21:39] <Maas-Maarten Zeeman> that the one doing the handoff has to wait for an ok to send more?
[11:21:52] <wok> they are ack'd - you need to know the result to update your administration
[11:22:19] <wok> so maybe we just answer with:
[11:22:37] <Maas-Maarten Zeeman> aha, so if you want to slow down the handoff you can also delay the ack?
[11:22:47] <wok> {ok, Result} | {error, Reason}
[11:23:19] <wok> where Reason could be: {busy, MinimumDelayInMsec} | WhateverElse
[11:23:45] <Maas-Maarten Zeeman> a yes.
[11:23:52] <wok> not delay too much, then the zynamo command FSM will be confused :p
[11:24:30] <wok> but we can give the FSM a timeout of (say) a couple of minutes, then the recipient can tarpit the sender
[11:24:37] <Maas-Maarten Zeeman> was wondering if you always had to delay.
[11:25:27] <Maas-Maarten Zeeman> but this allows for as fast as possible handoffs not bothered by minimum delays for every request.
[11:25:33] <wok> Then I make my handoff thingy in a separate process, which can be hung, without delaying any other handoff
[11:25:53] <wok> I like your tarpit idea, very elegant
[11:26:02] <wok> no extra logic needed :)
[11:26:26] <Maas-Maarten Zeeman> k
[11:26:54] <wok> Will do it that way, just have to split my handoff processes and add them to a supervisor.
[11:27:13] <wok> max 1 process per node
[11:27:38] <Maas-Maarten Zeeman> aha, no problems with synchronization of dbs? commits/rollbacks?
[11:27:52] <wok> eventual consistency FTW :p
[11:28:08] Thomas Spellman leaves the room
[11:28:19] <Maas-Maarten Zeeman> vector clocks.. still have to get used to that...
[11:28:35] <wok> Oh, timestamps work also quite well
[11:28:55] <wok> :) especially when you can trust your clock and have a low update frequency
[11:28:59] <Maas-Maarten Zeeman> now busy with sql layer... commit .. rollback...
[11:29:09] <wok> per node very much needed!
[11:29:22] <wok> consistent per node, eventual consistent over the whole system
[11:29:52] <Maas-Maarten Zeeman> ran zynamo a couple of days ok. looks very promising.
[11:30:48] <wok> is simple enough (after many iterations of the core code :)
[11:31:02] <wok> much simpler than Riak
[11:32:03] <Maas-Maarten Zeeman> That's the most difficult part isn't it?
[11:32:58] <wok> it is the most tricky part — have the stuff gossip and stable
[11:33:35] <wok> and deciding how to do it — that was very time consuming, but better think hard before doing something wrong.
[11:34:38] <Maas-Maarten Zeeman> zotonic also has a very different goal as riak :-)
[11:35:47] <wok> indeed, and because of that the hard decision to not use their stuff
[11:37:29] <Maas-Maarten Zeeman> I guess so.
[11:37:32] <wok> the live stats are really cool :)
[11:37:43] <Maas-Maarten Zeeman> Very nice to have something like riak as example though
[11:37:52] <wok> indeed :p
[11:37:59] <wok> Open source FTW
[11:38:54] <Maas-Maarten Zeeman> Will look into the stats later. Was busy wondering how I could straighten out error messages from different dbs.
[11:39:29] <Maas-Maarten Zeeman> just returning error is a possiblity :-)
[11:39:30] <wok> maybe not?
[11:39:43] <wok> indeed, tag it with the db-type
[11:40:01] <wok> then the user knows how to read the error
[11:40:18] <Maas-Maarten Zeeman> yeah
[11:40:40] <Maas-Maarten Zeeman> first things first. I also wait a bit with calls like prepare
[11:41:45] <Maas-Maarten Zeeman> something like q1 and q work pretty well and are simple to implement cross dbs.
[12:01:34] arjan leaves the room
[12:03:17] <wok> cool - Arjan just committed multiple nodes on one graph :)
[12:04:08] <wok> prepared statements are nice for performance - but that is something we can do later (and might be done in the db layer itself - keep stats which queries are executed often, prepare those)
[12:12:49] <Maas-Maarten Zeeman> Wow, that's quick...
[12:16:51] <Maas-Maarten Zeeman> Me not so quick :-) But what else to do with christmas holiday :-)
[12:40:58] <Maas-Maarten Zeeman> I guess I have to start shopping to get my hands on a simple (and cheap) dev cluster. :-)
[13:44:19] <Maas-Maarten Zeeman> Funny 'query' is a reserved word in Erlang syntax. It doesn't do anything. Relic from the past I guess.
[13:51:28] <Maas-Maarten Zeeman> huh, it is actually used :-) http://www.erlang.org/documentation/doc-5.5.5/pdf/mnemosyne-
[13:53:17] <wok> lots of people bump into 'query' - so silly to make that one reserved
[13:54:07] <wok> handoff is coming down nicely, now on to listen to the 'node up' signals to make sure that our handoff fsms are running.
[13:55:05] <Maas-Maarten Zeeman> How do you test all that stuff?
[14:28:13] arjan joins the room
[15:13:06] <wok> very careful reading of the source :p
[15:13:15] <wok> and actually most routines are small
[15:13:40] <wok> and we have a ets based kv store in zynamo, which is a great test case.
[16:28:58] Maas-Maarten Zeeman leaves the room
[17:15:14] maas.maarten.zeeman joins the room
[19:06:40] vio42 joins the room
[19:38:41] maas.maarten.zeeman leaves the room
[20:26:44] maas.maarten.zeeman joins the room
[23:11:27] Thomas Spellman joins the room
Powered by ejabberd Powered by Erlang Valid XHTML 1.0 Transitional Valid CSS!