Monday, 11 February 2013< ^ >
arjan has set the subject to: Zotonic - the Erlang Content Management Framework
Room Configuration

[07:59:35] maas.maarten.zeeman joins the room
[09:42:10] Andreas Stenius joins the room
[09:42:54] <Andreas Stenius> maas.maarten.zeeman: +1 :)
[09:44:00] <maas.maarten.zeeman> See also the folsom collector on my account... not tried yet. It requires one to actually define your counters and histograms, so the stats api needs an extension for that.
[09:46:16] <Andreas Stenius> Nice :)
[09:47:20] <Andreas Stenius> not sure what you had in mind for supporting the definition of stats in the api
[09:47:42] <Andreas Stenius> but I think it ought to be enough to have a call you can make to z_stats asking for what kind of statistics there are
[09:47:45] <maas.maarten.zeeman> Something like this? new_counter(Name, From)
[09:47:57] <Andreas Stenius> so the stats handler can define the stats needed, in case that is required..
[09:48:03] <maas.maarten.zeeman> It needs a model too I think
[09:48:32] <Andreas Stenius> to pull stats back for presentation? or to update stats from a template? :)
[09:48:51] <maas.maarten.zeeman> Yep, just the current numbers
[09:49:00] <Andreas Stenius> ah :)
[09:49:29] <maas.maarten.zeeman> Update stats would also be fun indeed.
[09:49:41] <maas.maarten.zeeman> So you can link it to your pages too.
[09:49:46] <maas.maarten.zeeman> hmm....
[09:49:57] <Andreas Stenius> I'd like to avoid the name clash of z_stats_handler though. I'd name them something like z_statman_handler and z_folsom_handler, then have a env config z_stats_handler pointing to the module it should use..
[09:49:59] <maas.maarten.zeeman> With a notification I think
[09:50:25] <Andreas Stenius> hmm.. but there's no context
[09:50:37] <Andreas Stenius> and we shouldn't look it up for each stat
[09:51:17] <maas.maarten.zeeman> No, but each site which has a context can easily count :-)
[09:51:52] <Andreas Stenius> maybe be nasty and have a little trick, loading the chosen stat handler under a common fictious name (z_stat_handler)
[09:52:59] <maas.maarten.zeeman> Maybe it is it possible to configure aliases in the code server. But that is nasty indeed.
[09:53:57] <maas.maarten.zeeman> Will check the speed of application:get_env first...
[09:54:06] <Andreas Stenius> I'm just sure that having every stats handler use the same module name will result in name conflicts and confusion
[09:54:18] <Andreas Stenius> good point :)
[09:54:27] <Andreas Stenius> just assumed it was a no-go there.. :p
[09:55:05] <Andreas Stenius> if it is fast enough, you could swap handler in a running system.. ;)
[09:55:19] <maas.maarten.zeeman> Indeed, yes.
[09:56:03] <Andreas Stenius> are there too many stats to have a single process channel all updates? I guess that'll become a bottleneck..
[09:57:33] <maas.maarten.zeeman> folsom and statman use parallel writes on an ets table. Works pretty quickly.
[09:57:39] <Andreas Stenius> hmm... how about that stats context we were thinking about? if we had such a context, we could collect stats in list in the context, and send it off at the end of a request, updating a whole bunch of stats in one go
[09:58:10] <Andreas Stenius> yeah, I was thinking if z_stats would be a server process, collecting stats, and passing them on to the handler
[09:58:35] <Andreas Stenius> in that case, the current handler could be stored in the z_stats process state..
[09:58:59] <maas.maarten.zeeman> Aha. Folsom had that but moved to a parallel ets based update setup
[09:59:15] <maas.maarten.zeeman> because the process became a bottleneck
[09:59:55] <Andreas Stenius> yeah, I remember seeing something about parallell-ism when it comes to updating the stats; I assume it is to not become a constraining factor on the system
[10:00:05] <maas.maarten.zeeman> Having a huge message box. I'm sure we have some bottlenecks in zotonic too :-p
[10:00:20] <Andreas Stenius> yep ;)
[10:00:34] <Andreas Stenius> so better not add another one :p
[10:01:20] <maas.maarten.zeeman> marc actually removed one a while ago... the sites dispatcher....
[10:01:34] <maas.maarten.zeeman> well, made it less of a bottleneck..
[10:01:35] <Andreas Stenius> did he?
[10:01:38] <Andreas Stenius> ah
[10:02:38] <maas.maarten.zeeman> All requests go through 1 gen_server though, but they are all io bound
[10:02:47] <Andreas Stenius> how about spawning a stats collector per request? that will be responsible for forwarding all stats to the stats handler
[10:03:10] <maas.maarten.zeeman> Lots of tricks are possible :-)
[10:03:14] <Andreas Stenius> ok, but from there it fans out, right?
[10:03:55] <Andreas Stenius> we need better stats, for the stats ;)
[10:04:39] <maas.maarten.zeeman> Having stats is the first step :-)
[10:05:12] <maas.maarten.zeeman> In the past I counted lines in the access logs..
[10:07:01] <maas.maarten.zeeman> Have seen it grow to 1.000.000 lines in an hour.
[10:07:58] <Andreas Stenius> whoho :p
[10:07:58] <maas.maarten.zeeman> This is a nice talk btw. http://www.infoq.com/presentations/Loopholes-CAP
[10:08:39] <Andreas Stenius> ah, yeah. I have it in a tab (from your comment on github :)
[10:17:01] <maas.maarten.zeeman> cool use gps time...
[10:44:31] <maas.maarten.zeeman> PACELC FTW!
[10:57:46] Andreas Stenius reading up on knockoutjs.com <- seems to be a cool library! :)
[10:59:28] <maas.maarten.zeeman> trying to fix a git conflict here, sigh.... clobbered stuff all over the place. where is hg update -C when you need it?
[11:01:18] <Andreas Stenius> hang on maas.maarten.zeeman.. what do you need :)
[11:01:30] <Andreas Stenius> what is hg update -C?
[11:01:45] <Andreas Stenius> is it to simply revert back to before you started?
[11:01:51] <maas.maarten.zeeman> locally I had the good stuff, and then I did a pull
[11:02:08] <maas.maarten.zeeman> ... and then git ate my homework
[11:02:17] <Andreas Stenius> was it committed?
[11:02:26] <maas.maarten.zeeman> local yes
[11:02:34] <Andreas Stenius> then we can get it back :D
[11:03:05] <Andreas Stenius> we just need to figure out where it is we want to go
[11:04:08] <Andreas Stenius> when you pulled, did it say something about rebasing, or applying your commits on top of head?
[11:04:18] <Andreas Stenius> or rewinding...
[11:04:30] <Andreas Stenius> (not sure about what it says, exactly)
[11:04:32] <maas.maarten.zeeman> I wanted to push, but that wouldn't let me
[11:05:11] <Andreas Stenius> ah, yes, it won't push unless it's a fast forward (or you give it a --force option, but it's good practive to only do fast-forward pushes)
[11:05:14] <maas.maarten.zeeman> so i did a pull first (assuming not much had changed anyway)
[11:05:45] <Andreas Stenius> and that pull gave you conflicts?
[11:06:10] <maas.maarten.zeeman> 4 conflicts, 3 easy, 1 not so easy
[11:06:27] <Andreas Stenius> do you want to go back, or resolve it?
[11:07:11] <maas.maarten.zeeman> I want to resolve it with the last local version
[11:07:15] <maas.maarten.zeeman> :-)
[11:07:26] <maas.maarten.zeeman> the file is full of crazy ass regexes....
[11:07:31] <Andreas Stenius> :)
[11:07:34] <Andreas Stenius> heh
[11:07:41] <maas.maarten.zeeman> recent xkcd ...
[11:08:29] <Andreas Stenius> xkcd?
[11:08:54] <maas.maarten.zeeman> http://xkcd.com/1171/
[11:09:33] <Andreas Stenius> heh :p
[11:09:58] <maas.maarten.zeeman> Can I do a checkout of the local version.
[11:10:13] <Andreas Stenius> sure
[11:10:31] <Andreas Stenius> I think the best thing is to go back, try: git merge --abort
[11:10:53] <maas.maarten.zeeman> what does that do abort the merge after the pull
[11:10:54] <Andreas Stenius> you will still have the upstream commits in your objects
[11:11:02] <Andreas Stenius> yep
[11:11:11] <maas.maarten.zeeman> k thanks.
[11:11:11] <Andreas Stenius> but you will get your local branch clean
[11:11:37] <maas.maarten.zeeman> few...
[11:11:39] <Andreas Stenius> git pull always try to merge with your upstream
[11:11:55] <Andreas Stenius> git fetch simply pulls in the changes without touching your stuff
[11:12:41] <maas.maarten.zeeman> most other repo's we have are mercurial.... there merging is separate
[11:12:57] <Andreas Stenius> but for the complicated merge... I don't think I have much advice for you besides using a good three way merge tool ;)
[11:13:10] <Andreas Stenius> I know :p
[11:14:30] <Andreas Stenius> gut git is nice in that way that as long as you have it committed, and preferably with a branch name pointing to the last commit, you can always recover from whatever screwed up state you find yourself in :)
[11:15:05] <Andreas Stenius> hint: create a dummy branch name pointing to your current master, so you have a ref to go back to (so you don't need to lookup the sha value) ;)
[11:15:27] <Andreas Stenius> if/when you're about to test something new
[11:15:53] <maas.maarten.zeeman> I'll merge in a separate directory I think. Should have done a pull before I started...
[11:16:07] <Andreas Stenius> maybe your conflict will be easier to apply ontop of the remote head, instead of merging the remote head directly onto your changes..
[11:16:14] <Andreas Stenius> in that case, a rebase might do it :)
[11:16:41] <Andreas Stenius> git rebase upstream/master master
[11:16:50] <maas.maarten.zeeman> I'll leave that for later today. Have somebody waiting for the change.
[11:16:59] <Andreas Stenius> lookup git help rebase in case you have more "interesting" rebasing scenarios :)
[11:17:20] <maas.maarten.zeeman> Have to finish the book I guess.
[11:17:24] <maas.maarten.zeeman> :-)
[11:17:30] <Andreas Stenius> ;)
[11:17:55] <Andreas Stenius> git rebase is really good at moving a set of commits around different branches...
[11:19:02] <Andreas Stenius> git rebase topic-a topic-b --onto master (pick changes made in topic-b, that branched off of topic-a, and apply them onto master, leaving any topic-a changes out of it :)
[11:19:04] <maas.maarten.zeeman> Feeling a bit like Donald Duck who painted himself stuck in the corner...
[11:19:49] <Andreas Stenius> :)
[11:20:02] <Andreas Stenius> you'll get it, just work with it :)
[11:21:09] <maas.maarten.zeeman> Or do a goofy workaround.
[11:30:20] maas.maarten.zeeman leaves the room
[12:14:34] maas.maarten.zeeman joins the room
[16:17:21] <Andreas Stenius> from: http://blog.stevensanderson.com/2012/08/01/rich-javascript-applications-the-seven-frameworks-throne-of-js-2012/
"All the technologies follow from the view that serious JavaScript applications require proper data models and ability to do client-side rendering, not just server rendering plus some Ajax and jQuery code."
And I think I can agree with this (note, this if for when building web apps, not simply serving pages/content). So, taking this to zotonic means a shift from the use of templates (erlydtl) to static html+js that binds the data to the page in the clients browser, updating it as the data changes.
[16:19:59] <maas.maarten.zeeman> or just $.html( rendered-on-server ) :-)
[16:21:51] <Andreas Stenius> I'd like the responsiveness of having the UI updated based on navigation on the page when all data is already loaded, instead of having to wait on a server roundtrip for rendering..
[16:22:10] <Andreas Stenius> granted, /all/ data is rarely available, but still...
[16:22:19] <maas.maarten.zeeman> that is nice for pre-defined controls.
[16:22:43] <maas.maarten.zeeman> with zotonic a roundtrip over ws to a web app on the other end of the globe is 80ms
[16:24:23] <Andreas Stenius> well, the $.html() is what we have now, when updating parts of the page with a postback returning a rendered template
[16:24:46] <Andreas Stenius> but if the data is rendered on the client, it is easier to merge in updates with local edits...
[16:25:38] <Andreas Stenius> hmm... or I'm just lacking some transparent data binding between the server and client state...
[16:25:39] <maas.maarten.zeeman> with channel i have a mix. using pre-rendered templates for js based updates
[16:25:55] <maas.maarten.zeeman> the browser knows best.
[16:26:00] <Andreas Stenius> lol-
[16:26:02] <maas.maarten.zeeman> (the state of the page that is)
[16:26:19] <Andreas Stenius> yeah, thus far ;)
[16:26:20] <maas.maarten.zeeman> What i do right now is have a communication bus
[16:26:30] <Andreas Stenius> ah, your message_bus
[16:26:38] <maas.maarten.zeeman> server can send messages over the bus which gets broadcasted on the page
[16:26:53] <maas.maarten.zeeman> (so is not targeted at a specific element)
[16:27:22] <maas.maarten.zeeman> it depends a bit on the message what happens.
[16:27:24] <Andreas Stenius> isn't that like your mod_signal?
[16:27:52] <maas.maarten.zeeman> you can do active things. ill post a snippet... :-)
[16:28:15] <Andreas Stenius> I'd like to make the data flow as transparent between client and server, as it does from data to view when using i.e. knockout
[16:30:28] <maas.maarten.zeeman> here is a ping-pong example...
[16:30:40] <maas.maarten.zeeman> /* ping.js */
;(function(channelme) {
function now() {
return new Date().getTime();
channelme.observe("/session/received/pong", function(e) {
var ping_time = now() - e[1];
channelme.notify("/bus_info", ping_time);
var ping_interval = setInterval(function() {
channelme.notify("/session/send_data", "{'ping' " + now() + "}$");
}, 5000);
[16:31:36] <Andreas Stenius> ah, this looks great :D
[16:32:01] <maas.maarten.zeeman> in another part i have this:
[16:33:11] <Andreas Stenius> is this in your message-bus branch ? or what's in there ?
[16:33:35] <maas.maarten.zeeman> no this is a sneak peak private code thingy.
[16:34:23] <Andreas Stenius> I suspected that fromt he channelme references.. ;) (but still, there *is* a message-bus branch :p )
[16:34:45] <maas.maarten.zeeman> yes, this uses a custom websocket.
[16:35:11] <maas.maarten.zeeman> allows you to send application data and also something else then js code
[16:35:17] <maas.maarten.zeeman> receive
[16:35:28] <Andreas Stenius> ah.. I remeber you committed some nifty websocket stuff a few months ago (if this is related to that?)
[16:35:44] <maas.maarten.zeeman> yes
[16:35:58] <maas.maarten.zeeman> need to make some parts more generic.
[16:36:52] <maas.maarten.zeeman> hmm i miss a template which i wanted to show.
[16:36:54] <Andreas Stenius> so, is this using the mod_bus module.. or is that work being done based on this?
[16:37:26] <maas.maarten.zeeman> part if this is in the message_bus branch on my zotonic clone
[16:38:04] <Andreas Stenius> as I understand it, the message bus could be used to shove data back and forth between the page and the server, right
[16:38:21] <maas.maarten.zeeman> Here is another snippet. I want to make things more generic. Everything is a bit verbose.
[16:38:23] <maas.maarten.zeeman> span id="bus_info">websocket starting</span>
{% javascript %}
channelme.observe("/bus_info", function(e) { $("#bus_info").html("Websocket ping:" + e); });
channelme.observe("/stream/open", function(e) { $("#bus_info").html("Websocket open"); });
channelme.observe("/stream/closed", function(e) { $("#bus_info").html("Websocket closed: " + e); });
channelme.observe("/stream/error", function(e) { $("#bus_info").html("Websocket error: " + e); });
{% endjavascript %}
// <span id="frame_info">frame starting</span>
{% javascript %}
var frame_check_interval = setInterval(function() {
var frame = window.frames["session-frame"];
if(frame) {
try {
$("#frame_info").html("Session frame ok");
$("#frame_info").html("No channelme found... it maybe loading...");
} catch(e) {
$("#frame_info").html("Could not access session frame: " + e);
} else {
$("#frame_info").html("No session frame found.");
}, 2000);
{% endjavascript %}
[16:40:04] <maas.maarten.zeeman> wrong template. :-/
[16:40:50] <maas.maarten.zeeman> it connects stuff from the bus to an info display widget thingy
[16:41:12] <Andreas Stenius> I think I get your idea with the message bus. It's like a publish/subscribe network
[16:41:28] <Andreas Stenius> for both ends (client/server)
[16:41:29] <maas.maarten.zeeman> it can be used like that.
[16:42:58] <Andreas Stenius> as can mod_signal..
[16:44:29] <maas.maarten.zeeman> I now mostly use mod signal on the server-side :-) mod_signal can't survive websocket failures and reconnects.
[16:44:43] <maas.maarten.zeeman> (needs work)
[16:44:43] <Andreas Stenius> ah
[16:44:56] <Andreas Stenius> reading up on your websocket changes...
[16:46:10] <maas.maarten.zeeman> zotonic postbacks and postback responses need to find a place in that :-)
[16:46:40] <maas.maarten.zeeman> with channel i also stopped using json. :-p
[16:46:47] <Andreas Stenius> ah, so if I use a custom ws handler, postbacks won't work..
[16:47:15] <maas.maarten.zeeman> for the moment I have 2 websockets.
[16:47:23] <maas.maarten.zeeman> :-p
[16:47:49] <Andreas Stenius> ah, one for postbacks and one for the message bus, I guess
[16:48:01] <Andreas Stenius> what do you use instead of json?
[16:48:30] <maas.maarten.zeeman> joe armstrong's ubf.
[16:48:52] <Andreas Stenius> aha... gotta go, youngest need to go poo... :p
[16:49:09] <maas.maarten.zeeman> number one has got prio :-) good luck
[16:57:26] <maas.maarten.zeeman> No this is definitely a number two.
[16:59:19] <Andreas Stenius> yeah, well, it was both number one and two, to be precise :p
[16:59:32] <maas.maarten.zeeman> thanks for the update ;-)
[16:59:52] <Andreas Stenius> heh :)
[17:00:32] <Andreas Stenius> ubf you say. I've looked at it briefly some time ago (years).. guess I'll have to revisit it..
[17:01:21] <maas.maarten.zeeman> well, not that important actually. just wanted to keep the data/meta-data a bit lower than found in json
[17:02:04] <maas.maarten.zeeman> no no {'blah', 100, 'foo', 200}
[17:02:16] <maas.maarten.zeeman> but {100, 200}
[17:02:38] <maas.maarten.zeeman> and have a record definition somewhere
[17:03:51] <maas.maarten.zeeman> we are sending over a lot of data
[17:07:18] <maas.maarten.zeeman> for zotonic it is better to use json i think
[17:11:56] <Andreas Stenius> ok
[17:12:25] <Andreas Stenius> dinner time... (constant stream of interruptions at this time of day... :p )
[17:12:50] <maas.maarten.zeeman> likewise over here. working from home today.
[17:13:16] <Andreas Stenius> I have my office at home... so this is a recurring setup for me... ;)
[17:27:29] maas.maarten.zeeman leaves the room
[21:46:10] maas.maarten.zeeman joins the room
[21:53:18] Jeff Bell leaves the room
[21:53:22] Jeff Bell joins the room
[21:55:21] Jeff Bell leaves the room
[22:20:21] Andreas Stenius leaves the room
[22:48:27] maas.maarten.zeeman leaves the room
Powered by ejabberd Powered by Erlang Valid XHTML 1.0 Transitional Valid CSS!