Friday, 8 March 2013< ^ >
arjan has set the subject to: Zotonic - the Erlang Content Management Framework
Room Configuration

[07:23:37] Maas joins the room
[08:22:12] Arjan joins the room
[08:44:24] Maas leaves the room
[08:49:32] Andreas Stenius joins the room
[08:49:51] Maas joins the room
[09:45:26] <Marc Worrell> I have to export some data as CSV. Anyone any ideas for something more generic? Maybe we need a mod_export….
[09:46:18] <Marc Worrell> (Also have a need for exports in vCal/iCal and vCard)
[09:46:20] <Andreas Stenius> what do you mean more generic?
[09:46:39] <Marc Worrell> like not copying/writing the whole controller again for another export
[09:47:15] <Marc Worrell> it is basically a stream of rows that is being reformatted and sent to the browser
[09:47:15] <Andreas Stenius> ah, like a pluggable thing you can pass rsc data to, and it spits it out in whatever format it was built for..
[09:47:46] <Marc Worrell> yep, or just rows, maybe even configuring the columns in the dispatch rule
[09:47:55] <Maas> So you can export a erlang table/proplist to whatever... Btw, just fixed the compile warning for ua_classifier.
[09:48:12] <Marc Worrell> :) @ compile warning
[09:48:14] <Andreas Stenius> yeah, I was thinking that the dispatch could direct to both which pluggable exporter to use, and options for it
[09:48:47] <Andreas Stenius> where some options may come from the request URI
[09:48:56] <Marc Worrell> yeah, or you make something generic that does some notifications, with maybe the name of the dispatch rule in the notification
[09:49:26] <Marc Worrell> Was thinking of that - then you can add some simple access control as welll
[09:49:37] <Andreas Stenius> indeed :)
[09:49:57] <Andreas Stenius> well, I'm thinking that it should be possible to use what we have already
[09:49:59] <Marc Worrell> Same for ical and vcard exports
[09:50:03] <Andreas Stenius> how about using templates for formatting the data
[09:50:16] <Andreas Stenius> and implement a model to take care of the collecting of data
[09:50:31] <Andreas Stenius> options can be passed as var bindings to the template
[09:50:49] <Maas> btw, csv can be surprisingly difficult to generate i remember.. all sorts of escaping and what not.
[09:50:54] <Marc Worrell> indeed
[09:51:00] <Andreas Stenius> we have all kinds of flexibility built in when it comes to templates :)
[09:51:02] <Marc Worrell> that is why I want to do it in code
[09:51:06] <Marc Worrell> also for dates etc
[09:51:37] <Marc Worrell> it could be a lot of data though, so I would prefer to stream it.
[09:51:41] <Marc Worrell> or chunk it
[09:51:42] <Andreas Stenius> for the tricky stuff, we could rely on filters... or let the model "know" how we want it formatted, like for dates
[09:51:54] <Marc Worrell> escape_csv
[09:51:58] <Andreas Stenius> hmm...
[09:52:01] <Maas> You probably already know about the python csv api?
[09:52:07] <Marc Worrell> nope
[09:52:15] <Andreas Stenius> me either
[09:52:20] <Maas> http://docs.python.org/2/library/csv.html
[09:52:31] <Maas> A stream api for reading and writing
[09:52:32] <Andreas Stenius> but still, I think templates are a good fit
[09:52:42] <Andreas Stenius> can stream the data using templates too
[09:52:44] <Marc Worrell> for small data they are
[09:52:55] <Marc Worrell> just have to be very careful with your newlines :p
[09:53:08] <Andreas Stenius> true ;)
[09:53:33] <Marc Worrell> but then most queries for these exports are code anyway - so it is ok to hook into some notifications
[09:53:46] <Maas> Some formatting details too. http://docs.python.org/2/library/csv.html#csv-fmt-params
[09:54:18] <Maas> iolists are your friend.
[09:54:48] <Marc Worrell> generating is easier than parsing - so much is clear about csv :p
[09:55:29] <Andreas Stenius> dispatch -> controller -> notification -> exporter; loops over rows -> template fragment -> stream rsp
[09:56:13] <Andreas Stenius> so the template is responsible for a single row at a time
[09:56:27] <Marc Worrell> and then at first without the template, because adding 20 columns to a single line template is maybe a tad hard to edit...
[09:56:29] <Andreas Stenius> and the exporter can stream back the data at one rows chunks
[09:57:02] <Andreas Stenius> or you filter out newlines from the row
[09:57:02] <Marc Worrell> or the template should do newlines between the columns, and then the exporter reformats those newlines
[09:57:18] <Andreas Stenius> yeah
[09:57:26] <Marc Worrell> still not sure though :p
[09:57:39] <Andreas Stenius> heh :p
[09:58:02] <Maas> Are there csv libs for erlang?
[09:58:04] <Marc Worrell> Just make a default without the template and then later add it as an option when you want to post-process data from the producer
[09:58:25] <Marc Worrell> Generating CSV is not the hard part :p
[09:58:37] <Andreas Stenius> no, indeed it is not. I was going for the generic
[09:58:51] <Andreas Stenius> where you may want to have the data in a what-not-format from the future ;)
[09:59:21] <Marc Worrell> yes, but that should be easy to add into the data processing pipe line.
[09:59:24] <Andreas Stenius> would be nice if most cases could be solved simply by overriding the row template
[10:00:04] <Marc Worrell> mod_export -> controller_export_rows ? :p
[10:00:19] <Marc Worrell> can also specialize them, easier at first
[10:00:20] <Andreas Stenius> sounds good :)
[10:00:38] <Marc Worrell> Some formats are really tricky with their start/stop sequences
[10:00:42] <Andreas Stenius> well, why not the best of both worlds
[10:00:46] <Andreas Stenius> as you were on to
[10:00:49] <Andreas Stenius> use notification
[10:01:00] <Andreas Stenius> the default could hook into those and use code for csv
[10:01:07] <Marc Worrell> yep
[10:01:18] <Marc Worrell> at least keep it loosely couple, then we can adapt it
[10:01:18] <Andreas Stenius> another with higher prio could hook into them and call a template or what not instead
[10:01:34] <Andreas Stenius> if enabled..
[10:03:10] <Marc Worrell> when you have multiple vcards then it is also just appended, isn't it? No extra header/footer
[10:03:44] <Marc Worrell> put it into mod_export or something else?
[10:03:55] <Marc Worrell> maybe we want mod_import_export….
[10:04:26] <Andreas Stenius> do they share functionality? otherwise I'd say it's tidier with mod_import and mod_export
[10:04:30] <Marc Worrell> and add import stuff there as well, handling uploads of csv files, that kind of stuff, or posts/puts of csv files to a controller
[10:04:41] <Marc Worrell> I think import is way more complex
[10:04:44] <Andreas Stenius> hmm... ok, that's a fair point
[10:04:45] <Marc Worrell> so better split it
[10:05:03] <Andreas Stenius> there's never only one way to do anything :p
[10:05:21] <Marc Worrell> and you might not want to enable any import in your site anyway - less hacking surface
[10:05:28] <Andreas Stenius> indeed
[10:05:36] <Andreas Stenius> +1 for splitting
[10:05:44] <Marc Worrell> keeping code tidy and loosely couple is important
[10:06:13] <Marc Worrell> Then I will make a mod_export, start with csv and also add ical and vcard stuff there
[10:06:28] <Andreas Stenius> cool :)
[10:06:54] <Marc Worrell> (need this stuff for maxclass - lots of data needs to be shared - but almost all data is generated on the site)
[10:07:09] <Maas> Used to have an exporter on another site. Default it generated html tables (with paging), it supported export via or accept-encoding header. qs format=csv. Then you got the same table in another format.
[10:07:39] <Marc Worrell> mod_export could hook into the controller_id for that kind of exports
[10:08:07] <Marc Worrell> the idea is that you go to /id/123 with an accept-encoding and that one 303s you to the correct controller
[10:08:21] <Marc Worrell> (non informational resource vs informational resource)
[10:08:50] <Marc Worrell> that is why the "official" resource uri of all resources is like http://example.org/id/123
[10:09:33] <Maas> In this case it was a human redirect... someone clicking on a link :-)
[10:09:58] <Marc Worrell> :) you also want to do that - that is why you have a separate controller for each format
[10:10:21] <Marc Worrell> (especially for IE, that one really doesn't know its accept-encodings)
[10:12:05] <Maas> *
[10:12:57] <Marc Worrell> ok, adding mod_export - later for your review on a GitHub near you
[10:13:09] <Maas> Cool.
[10:13:50] <Maas> Maybe this can help you. https://github.com/refuge/ecsv
[10:14:43] <Marc Worrell> :) thanks!
[10:25:06] <Marc Worrell> Hmmm, would be nice when we could add some inheritance to the controllers.
[10:25:25] <Marc Worrell> -controller_extends(controller_blah).
[10:25:47] <Andreas Stenius> yeah, doesn't erlang support that already?
[10:25:55] <Maas> So you can mix-in stuff.
[10:26:04] <Maas> include...
[10:26:12] <Marc Worrell> Erlang did support that experimentally - removed in R16
[10:26:16] <Andreas Stenius> https://github.com/zotonic/zotonic/issues/278
[10:26:26] <Marc Worrell> but our webzmachine can do it easily.
[10:26:29] <Andreas Stenius> oh, did they..
[10:28:38] <Marc Worrell> yeah, R16 cleans up quite a bit - parametrized modules are out as well :)
[10:28:43] <Marc Worrell> never liked them anyway
[10:28:53] <Andreas Stenius> yeah, I know about the parameterized got out..
[10:28:58] <Andreas Stenius> never used them
[10:29:07] <Andreas Stenius> indeed, it felt a bit clumsy
[10:30:03] <Andreas Stenius> better not use extends either.. :p
[10:30:19] <Maas> so now we go roll one of our own :-)
[10:30:30] <Andreas Stenius> +1 :D
[10:30:35] <Marc Worrell> :p
[10:31:19] <Andreas Stenius> for a bunch of guys in favor of "not invented here", I hear a lot of: "let's make our own" or.. "ok, let's fork that.." .. :p
[10:32:01] <Maas> :D
[10:33:08] <Maas> That that directive was used so much was sure a sign that there is a need for easily extending modules.
[10:34:05] <Maas> On the other hand, I also like the no-surprises way of erlang...
[10:36:38] <Marc Worrell> Yeah, problem with the Erlang module extension mechanism is that it is experimental.
[10:37:07] <Marc Worrell> And afaik there is no api to get the whole combined list of exports from a module - which we need...
[10:38:08] <Maas> Please don't use a parse transform for this..
[10:38:13] <Maas> ;-)
[10:38:52] <Marc Worrell> never :p
[10:39:12] <Marc Worrell> (will confuse the hell out of the build process anyway - hidden dependencies…)
[10:39:51] <Maas> Saw that yesterday with the updated lager. An update in the transform means compile everything... no simple live code upgrade.
[10:40:32] <Maas> Maybe there is a nice erlangy way to do module extensions/mixins.
[10:47:08] <Andreas Stenius> Marc Worrell: foo:module_info(exports)
[10:47:40] <Marc Worrell> yeah, I know that one - but it doesn't aggregate over the extended modules, does it?
[10:48:42] <Andreas Stenius> oh, I missed your previous line, thus that you were referring to the exports in conjunction with the extends directive..
[10:49:04] <Marc Worrell> yeah - webzmachine needs to know that, for the default fallbacks
[10:49:19] <Marc Worrell> bu then, every controller could extend the defaults...
[10:49:28] <Andreas Stenius> but if webzmachine is to support extends by itself, that will solve itself any way
[10:49:37] <Andreas Stenius> precisely
[10:49:59] <Maas> Maybe that could be done with the help of a behaviour.
[10:50:11] <Andreas Stenius> uh?
[10:50:22] <Maas> Like gen_server.
[10:50:36] <Andreas Stenius> behaviour's only make sure you export a defined set of functions.
[10:51:45] <Maas> Sorry I get a call.
[11:01:15] <Marc Worrell> %% @doc mod_export - return the content type (like "text/csc") for the dispatch rule/id export.
-record(export_resource_content_type, {
dispatch :: atom(),
id :: integer()
%% @doc mod_export - Fetch the header for the export.
%% The 'first' notification should return: {ok, binary()} | {ok, binary(), ContinuationState} | {error, Reason}.
-record(export_resource_header, {
dispatch :: atom(),
id :: integer()
%% @doc mod_export - fetch a row for the export, can return a list of rows, a binary, and optionally a continuation state.
%% The 'first' notification should return: {ok, Values|binary()} | {ok, Values|binary(), ContinuationState} | {error, Reason}.
%% Where Values is [ term() ], i.e. a list of opaque values, to be formatted with #export_resource_format.
%% Return the empty list of values to signify the end of the data stream.
-record(export_resource_data, {
dispatch :: atom(),
id :: integer(),
state :: term()
%% @doc mod_export - Format a single proplist of data.
%% The 'first' notification should return: {ok, binary()} | {ok, binary(), ContinuationState} | {error, Reason}.
-record(export_resource_format, {
dispatch :: atom(),
id :: integer(),
data :: term(),
state :: term()
%% @doc mod_export - Fetch the footer for the export. Should cleanup the continuation state, if needed.
%% The 'first' notification should return: {ok, binary()} | {error, Reason}.
-record(export_resource_footer, {
dispatch :: atom(),
id :: integer(),
state :: term()
[11:01:24] <Marc Worrell> something like this? :p
[11:03:30] <Andreas Stenius> yah, looks flexible enough :)
[11:04:16] <Andreas Stenius> and as we have the context along with the notification too, we can get at the q's too, right..
[11:10:06] <Marc Worrell> Indeed, then those observers can just check the q arguments, if they want to know something more.
[11:10:23] <Marc Worrell> And then we just add some default behaviour for csv etc
[11:10:45] <Marc Worrell> Like if the id is a collection/search - then export the items in the collection/search
[11:10:53] <Andreas Stenius> +1
[11:11:40] <Marc Worrell> And the access control for id should be enough - for now I just return 404 - no fancy redirects, this should be API like
[11:12:57] <Andreas Stenius> should do it, yep :)
[11:13:53] <Marc Worrell> nice, then today we will have a vehicle for easier exports of data
[11:15:09] <Andreas Stenius> reading up on go (tour.golang.org).. funky looking syntax.. but getting over my scepticism for the whole thing, I kind of understand their design choices
[11:15:35] <Andreas Stenius> they have a few nice feats
[11:15:39] <Marc Worrell> yes, though some things are strange in the language - like no pattern matching
[11:15:52] <Marc Worrell> it is a kind of low level steam processing, isn't it?
[11:15:55] <Marc Worrell> stream
[11:16:52] <Andreas Stenius> yeah, I can't go any were now without missing pattern matching and the "don't touch my bound values, god d-mn it!" :p
[11:17:07] <Andreas Stenius> stream processing? go? dunno...
[11:17:32] <Marc Worrell> BTW the lager patches are accepted and merged into the master
[11:17:43] <Andreas Stenius> nice! :)
[11:18:37] <Marc Worrell> I also can't live anymore with destructive assignments into some shared objects… gets very hard for maintenance and debugging
[11:19:41] <Maas> Reading the go faq.. Funny question: Why is my nil error value not equal to nil?
[11:19:58] <Andreas Stenius> lol
[11:20:02] <Marc Worrell> huh?
[11:20:09] <Maas> Is this the new vb?
[11:20:17] <Andreas Stenius> guess nil doesn't compare at all
[11:20:29] <Andreas Stenius> which sounds stupid
[11:20:49] <Maas> No, every value is typed... so that means that every type has a nil value.
[11:21:22] <Maas> So much for pragmatism...
[11:21:49] <Marc Worrell> that nil thing is silly indeed - suddenly you have to deal with the intrinsics of the implementation….
[11:22:12] <Marc Worrell> so {int, nil} != {float, nil} ?
[11:22:19] <Andreas Stenius> oh, no!
[11:23:27] <Andreas Stenius> hehe... what about side effects in the case caluse!?
For example,switch i { case 0: case f(): }does not call f if i==0.
[11:23:38] <Andreas Stenius> clause*
[11:24:28] <Marc Worrell> huh??? wtf?
[11:24:57] <Maas> makes you wondering what the problem was that they set out to solve with go. It looks indeed like a new low-level programming language.
[11:25:19] <Andreas Stenius> it is quite low level.
[11:25:30] <Andreas Stenius> it like a redesigned C, with closures and stuff...
[11:26:03] <Marc Worrell> yes, and a lot more strange things you need to know
[11:26:14] <Marc Worrell> there are quite a bit of gotchas in there
[11:26:22] <Andreas Stenius> I think they were bothered with the C syntax ( http://golang.org/doc/articles/gos_declaration_syntax.html ) and made a new language with some extra frills to it
[11:27:03] <Maas> Right, extra details like closures running as goroutines.
[11:27:39] <Andreas Stenius> go'ey (as in slimy, muddy..) :p
[11:27:44] <Maas> Well, fp ftw!
[11:28:06] <Andreas Stenius> +1
[11:28:25] <Marc Worrell> yep - happy to use fp
[11:29:10] <Maas> And c when you want to be able to shoot yourself in your foot.
[11:30:50] <Andreas Stenius> know of any fp suitable to run on embedded hardware with limited resources (Kb rather then Mb) ?
[11:30:58] <Marc Worrell> they should just have user Erlang to begin with…. and then put some effort in beam optimizations….
[11:31:00] <Andreas Stenius> supporting ARM7..
[11:31:01] Arjan leaves the room
[11:31:09] <Marc Worrell> @Andrea: ML / CAML ?
[11:31:12] <Marc Worrell> +s
[11:31:33] <Maas> yeah (oca)ml is great.
[11:31:34] <Andreas Stenius> will have to look it up :)
[11:31:47] <Marc Worrell> https://github.com/bmeurer/ocaml-arm
[11:31:56] <Andreas Stenius> wow, cool
[11:32:14] <Maas> Compilers can do wonderful things with fp languages.
[11:32:18] <Marc Worrell> maybe also nice: https://ocaml.janestreet.com/?q=node/110
[11:32:37] <Marc Worrell> ocaml is nice - ML is very pragmatic and quite easy to compile well
[11:33:56] Arjan joins the room
[11:34:08] <Maas> ocaml goes both ways with vm and also native compiled stuff. First introduced to it via http://coq.inria.fr
[11:35:35] <Andreas Stenius> ow, the talk of moderate memory needs, and code size savings relative to some other version... no hard numbers any were! :p
[11:37:10] <Maas> For some proofs it could generate working ocaml programs. Which was kind of mind blowing at the time I took applied logic course.
[11:38:09] <Andreas Stenius> hmm.. googling embedded functional programming turned up quite a number of hits, actually :)
[11:38:30] <Maas> Last talk at functional cross over included a talk by someone which used a haskell based language to specify fpga's.
[11:40:51] <Andreas Stenius> interesting... http://en.wikipedia.org/wiki/Hume_(programming_language)
[11:41:24] <Maas> If you use ml based languages the type inference takes a little bit of time to adjust to. Sometimes quite a bit of head scratching too.
[11:45:08] <Maas> Hume, never heard of it. Functions bounded in space and time.
[11:45:24] <Maas> for real time embedded.
[11:45:32] <Andreas Stenius> the syntax looked super strange, though
[11:45:45] <Maas> A bit haskelly
[11:46:44] <Maas> No curly braces... Yeah!
[11:47:40] <Maas> There are a lot of programming languages out there. The strangest I used was http://en.wikipedia.org/wiki/Occam_(programming_language)
[11:47:52] <Maas> But that was also targeted at a strange computer.
[11:47:57] <Maas> :-)
[11:48:24] <Maas> or should I say transputer.
[11:49:22] <Andreas Stenius> heh ;)
[11:49:44] <Andreas Stenius> here's a strange mix: http://forthfreak.net/index.cgi?FunForth
[11:54:12] <Maas> You also have http://en.wikipedia.org/wiki/Embeddable_Common_Lisp
[11:56:52] <Maas> So much to choose from...
[11:58:19] <Andreas Stenius> that sounds neat... also make's it easier to convert from C...
[12:18:23] Maas leaves the room
[12:34:57] Maas joins the room
[12:58:49] Maas leaves the room
[13:00:35] Maas joins the room
[14:37:33] Maas leaves the room
[14:38:41] Arjan leaves the room
[14:38:49] Maas joins the room
[15:32:04] Maas leaves the room
[15:33:13] Maas joins the room
[16:33:26] <Maas> Cool... sqlite nif is used by others... got a patch for long long integers :-)
[16:33:43] <Andreas Stenius> :)
[16:55:20] Maas leaves the room
[16:58:37] <Andreas Stenius> Marc Worrell: check this out: http://stackoverflow.com/a/15298078/444060
[16:59:46] <Andreas Stenius> been searching all afternoon for that. I just knew I've read about it some where but couldn't find it... until now :)
[17:00:24] Maas joins the room
[17:02:06] <Andreas Stenius> also noting that ther's a erlang:function_exported(M, F, Arity) -> bool()... ;)
[17:02:29] <Marc Worrell> yep :)
[17:03:17] <Marc Worrell> argghhh $handle_undefined_function
[17:03:26] <Marc Worrell> is that documented somewhere?
[17:03:54] <Andreas Stenius> uh.. don't think so, process_flag(error_handler) is, sort of almost documented
[17:04:06] <Andreas Stenius> https://github.com/erlang/otp/blob/maint/lib/kernel/src/error_handler.erl#L139
[17:04:16] <Marc Worrell> LOL almost….
[17:04:40] <Andreas Stenius> http://erlang.org/doc/man/erlang.html#process_flag-2
[17:05:19] <Maas> LOL documentation...
[17:05:21] <Andreas Stenius> so, the process_flag call is documented, but not the api that the error_handler needs to provide
[17:05:44] <Andreas Stenius> "This is used by a process to redefine the error handler for undefined function calls and undefined registered processes. Inexperienced users should not use this flag since code auto-loading is dependent on the correct operation of the error handling module."
[17:06:12] <Marc Worrell> or, in other words: "HANDS OFF"
[17:06:13] <Marc Worrell> :p
[17:06:49] <Maas> Cool. Do NOT touch the red button...
[17:06:53] <Andreas Stenius> yeah.. I'd steer clear of changing the error_handler, but I think the $handle_undefined_function is useful
[17:07:26] <Andreas Stenius> could actualy make the inheritance stuff kind of generic with that
[17:07:54] <Andreas Stenius> include a hrl file that defines the handler, looks at the extends attribute of the module and voila
[17:08:33] <Andreas Stenius> yeah, Maas, I know it's hiding it with magic... but still ;P
[17:08:45] <Maas> It could be done.
[17:09:08] <Andreas Stenius> I'll make a proof of concept in z_stdlib... :D
[17:09:17] <Andreas Stenius> (in a branch, no worries)
[17:11:47] <Andreas Stenius> should add to the upgrade notes for 0.10 that you should do rm -rf deps/* && make get-deps
[17:25:36] Maas leaves the room
[17:34:25] Arjan joins the room
[17:40:40] Arjan leaves the room
[18:07:46] Andreas Stenius leaves the room
[18:12:11] Andreas Stenius joins the room
[19:11:14] Maas joins the room
[19:11:28] Maas leaves the room
[19:25:44] Andreas Stenius leaves the room
[19:34:52] Andreas Stenius joins the room
[21:06:04] <Marc Worrell> Added the more-or-less generic export module. As discussed :)
[21:06:24] <Marc Worrell> And now some docs…. arggh
[21:06:32] simon.smithies joins the room
[21:14:26] <simon.smithies> evening all
[21:15:03] <simon.smithies> andreas: am trying to get gitolite going along the lines you suggested a few days ago
[21:16:11] <simon.smithies> am trying to do it just with a post-receive hook -- everything is working fine except I can't change ownership of the updated files back to zotonic
[21:17:36] <simon.smithies> do you avoid all that by using mod_github or something in zotonic to pull from the repo, instead of a hook?
[21:21:42] Arjan joins the room
[21:37:22] Maas joins the room
[21:39:13] <Maas> Nice marc, was just tinkering with non-computer hardware... also fun once in a while.
[21:48:38] Maas39285 joins the room
[21:48:38] Maas39285 leaves the room
[21:48:38] Maas58766 joins the room
[21:48:38] Maas58766 leaves the room
[21:48:38] Maas12120 joins the room
[21:48:38] Maas12120 leaves the room
[21:48:38] Maas95625 joins the room
[21:55:36] Maas leaves the room
[21:57:25] <Arjan> how can I create a JS file with a template which gets cached by the browser in zotonic?
[21:57:43] <Arjan> e.g. a lookup table which needs to be cached clientside
[21:58:08] <Maas95625> Static js or from a template?
[21:58:13] <Andreas Stenius> isn't there any options for cache headers in controller_template... ?
[21:58:17] <Arjan> from a template
[21:58:25] <Arjan> there is no cache option in controller_template, no
[21:58:33] <Arjan> it just always returns a 200
[21:58:34] <Andreas Stenius> maybe we need to add some :p
[21:58:37] <Arjan> :)
[22:00:52] Arjan leaves the room
[22:01:38] <Maas95625> Watching swedish crimi, and programming erlang.
[22:02:03] <Maas95625> Takes place in Gotland....
[22:04:09] Arjan joins the room
[22:04:27] <Arjan> :)
[22:05:05] <Arjan> I added a {maxage, X} option
[22:05:14] <Arjan> to controller_template
[22:06:16] <Maas95625> Nice that you can cache it now. I server static js files that way, stringed together like the lib tag.
[22:06:45] <Maas95625> Sometimes it get's cached a bit too aggressive... :-)
[22:07:30] <Arjan> yeah
[22:07:41] <Arjan> I have i18n stuff in that JS file
[22:07:48] <Arjan> probably will break if I switch language
[22:07:59] <Arjan> or I should add the language to the URL, actually
[22:08:02] <Arjan> :P
[22:08:29] <Arjan> uhoh, gonna pull
[22:08:39] Arjan prepares himself for the rebar deps
[22:08:51] <Marc Worrell> maas, what did you use instead of now()?
[22:09:08] <Maas95625> os:timestamp... now locks.
[22:09:22] <Marc Worrell> huh? os:timestamp now locks???
[22:09:28] <Andreas Stenius> Arjan: no biggie, probably have to flush and reget your deps and it should be fine :)
[22:09:32] <Maas95625> now() locks the scedulers.
[22:09:42] <Marc Worrell> I see a now() in webzmachine - will replace it
[22:09:46] <Maas95625> to get unique timestamp.
[22:10:20] <Maas95625> If you call it a lot erlang will speed up the time... :-)
[22:10:32] <Marc Worrell> there are two now() calls for every webzmachine request - am replacing them
[22:10:45] <Arjan> Marc Worrell: looking forward to your docs for mod_export :P
[22:11:16] <Maas95625> for seeding the rng (not that it causes locks) I took the suggestion from lyse
[22:11:22] <Marc Worrell> Uh oh - three now()'s
[22:11:26] <Marc Worrell> also one for the logger :-s
[22:11:27] <Maas95625> use crypto:rand_bytes()
[22:11:30] <Maas95625> :-)
[22:11:36] <Andreas Stenius> uh, the $handle_undefined_function was added two months ago... doh. Need R16 for that...
[22:11:43] <Maas95625> I changed them in my stats branch... need to push that...
[22:11:50] <Maas95625> now I have conflicts....
[22:11:59] <Maas95625> :-p
[22:12:01] <Marc Worrell> :p
[22:12:16] <Andreas Stenius> conflict z_stats -> master, or ?
[22:12:21] <Marc Worrell> @Arjan - docs will be coming asap :p
[22:12:27] <Maas95625> No biggy for the conflicts.
[22:12:29] <Andreas Stenius> +1 :)
[22:12:31] <Andreas Stenius> ok
[22:13:00] <Arjan> :)
[22:13:00] <Andreas Stenius> the generic inheritance isn't that scare, actually... it's a tiny hrl file
[22:13:09] <Andreas Stenius> -export(['$handle_undefined_function'/2]).
'$handle_undefined_function'(F, A) ->
Inherits = proplists:get_value(inherits, module_info(attributes), []),
apply_inherits__(Inherits, F, A),
{undefined, {?MODULE, F, A}}.
apply_inherits__([], F, A) ->
{failed, {?MODULE, F, A}}; %error_handler:raise_undef_exception(?MODULE, F, A);
apply_inherits__([M|Inherits], F, A) ->
case erlang:function_exported(M, F, length(A)) of
true ->
apply(M, F, A);
false ->
apply_inherits__(Inherits, F, A)
[22:13:26] <Maas95625> There was one now in webmachine id too. Maybe that shoud be done a bit different too.
[22:13:26] <Andreas Stenius> oh, there's a bit of debugging tidbits in there...
[22:13:38] <Marc Worrell> argggg, the logger of webzmachine had three more now() calls....
[22:13:49] <Maas95625> Yup :-)
[22:13:55] <Maas95625> Locking all the way.
[22:14:22] <Maas95625> Probably causes an enormous speedup :-p
[22:14:41] <Andreas Stenius> then you can use it like:
-inherits([m_b, m_a]).
-export([f1/0, f4/0]).
f1() ->
f4() ->
[22:14:47] <Marc Worrell> we keep one - for the request id, that one needs to be unique...
[22:14:58] <Andreas Stenius> I'm off for the night...
[22:15:05] <Andreas Stenius> happy merging and stuff :)
[22:15:17] <Maas95625> Yes, was thinking about changin that with phash a make_ref an os:timestamp.
[22:15:18] <Marc Worrell> @Andreas - can I also ask that thing if it knows a function?
[22:16:23] <Maas95625> Btw, if now gets called a lot erlang will speed up time... if the vm then crashes and restarts you can get (theoratically) the same ids...
[22:16:41] <Marc Worrell> @Maas - you don't need to commit webzmachine - unless you have other changes than the now() calls?
[22:16:59] <Maas95625> Yes, I move the metadata to the logger record.
[22:17:08] <Andreas Stenius> Marc Worrell: easily added.. :)
still haven't tested enough, though, as I need to update my erl release to R16... :p
[22:17:15] <Maas95625> So I could do the couting at the zotonic side.
[22:17:42] <Maas95625> It knows what juicy parts where added.
[22:18:01] <Maas95625> Small change.
[22:18:05] Andreas Stenius signs off...
[22:18:09] Andreas Stenius leaves the room
[22:19:59] <Maas95625> https://github.com/mmzeeman/webzmachine/commit/051071460640f26621e0794ac2945c231985376f
[22:20:24] simon.smithies leaves the room
[22:22:36] <Maas95625> Adding the patch, it only contains adding 64 bit integers to sqlite, not taking them out again. Really nice though to have actual users of the library.
[22:22:41] <Marc Worrell> @Maas - that one is very mergeable still
[22:23:03] <Maas95625> Aleph Archives is using it.
[22:23:14] <Marc Worrell> your sqlite?
[22:23:19] <Maas95625> yes.
[22:23:24] <Marc Worrell> :)
[22:24:03] <Maas95625> More erlang users.
[22:24:11] <Marc Worrell> just removed the parametrized webmachine_controller
[22:24:38] <Maas95625> Wow, don't you have anything else to do... Just replaced a bearing on my rowingbike..
[22:25:13] <Marc Worrell> nah - it still has all the parametrized MochiWeb stuff in there....
[22:25:27] <Marc Worrell> this is a small change for the module extend stuff
[22:25:41] <Marc Worrell> but then we can just wait for R16… :p
[22:26:13] <Maas95625> Haha... mochiweb has it all over.
[22:27:28] <Marc Worrell> yes, one big pile of parametrized modules....
[22:29:16] <Maas95625> O right, you can put in a 64 bit integer, but you can't find out if the value you are getting is actually 64 bits...
[22:33:21] <Maas95625> Oh, nice when interfaces just connect like magic...
[22:33:36] <Maas95625> case SQLITE_INTEGER:
return enif_make_int64(env, sqlite3_column_int64(statement, i));
[22:36:04] <Maas95625> It doesn't even need a cast...
[22:36:52] <Marc Worrell> :)
[22:37:57] <Maas95625> Adding contributers file... (Did you copy the zotonic header from a google repo?)
[22:39:02] <Marc Worrell> yep
[22:39:13] <Marc Worrell> or some doc about how they do it
[22:40:10] <Maas95625> :-) You could read it in a way that it looks like that google holds the copyrights for zotonic.
[22:40:33] <Maas95625> # This is the official list of people who contribute
# (and typically have contributed) code to the esqlite repository.
# The AUTHORS file lists the copyright holders; this file
# lists people. For example, Google employees are listed here
# but not in AUTHORS, because Google holds the copyright.
[22:41:08] <Maas95625> But luckily it says "for example"
[22:46:31] Arjan leaves the room
[22:47:19] <Maas95625> Wow, it looks so easy now.
[22:53:15] <Marc Worrell> your code?
[22:53:16] <Marc Worrell> :p
[22:53:44] <Maas95625> No the webmachine controller without the parameterized stuff
[22:53:59] <Marc Worrell> :-) yeah, a lot less magic
[22:56:15] <Marc Worrell> maybe we should also rename all the modules - to prevent name clashes with Basho
[22:56:37] <Marc Worrell> webmachine_ … —> wzm_ ….
[22:57:19] <Maas95625> Yes indeed, it is starting to get very different.
[22:59:20] <Marc Worrell> need to check their repo - to see if there are any bug fixes that affect us
[22:59:54] <Maas95625> Indeed that could easily be the case.
[23:04:02] <Maas95625> There is also a parameterized module fix :-)
[23:04:42] <Maas95625> That one which you removed long time ago I think.
[23:06:45] <Marc Worrell> https://github.com/basho/webmachine/commit/ffc8dfb3034cdfbc42a5c91f10cb00e4e0fd517f
[23:07:34] <Maas95625> Yup, that one.
[23:07:52] <Marc Worrell> yep - did that three years ago
[23:08:17] <Marc Worrell> they still copy the whole dispatch lists, now from an ets table....
[23:09:34] <Maas95625> Good enough for smallish technical web interfaces probably.
[23:09:49] <Marc Worrell> yep - disaster for our use
[23:12:56] <Maas95625> It will be interesting removing mochiweb ...
[23:13:06] <Marc Worrell> yep
[23:13:17] <Maas95625> Lots of references... and small little details
[23:13:27] <Marc Worrell> was also thinking that we could combine the ReqData and Context into one
[23:13:33] <Marc Worrell> makes all controllers easier
[23:14:04] <Maas95625> So you can remove the macro's
[23:14:15] <Marc Worrell> as the first field of all #context{} could be just the reqdata
[23:14:30] <Marc Worrell> yeah - less stuff packing in and out
[23:15:03] <Marc Worrell> Will write it down as a note somewhere - before I forget
[23:16:04] <Maas95625> Moving to elli... it has a little bit different interface too.
[23:20:30] <Marc Worrell> not so many fixes in webmachine… as far as I can see
[23:20:44] <Marc Worrell> they added a stream option with known length
[23:20:57] <Marc Worrell> content-md5 header support
[23:21:39] <Marc Worrell> and an event handler for logging (not so good - always copies all messages to the event handler, even when nobody is logging...)
[23:27:16] <Maas95625> Fun, browsing through the commit log...
[23:27:47] <Marc Worrell> https://github.com/zotonic/webzmachine/pull/2
[23:28:12] <Marc Worrell> yep - shows what they are doing quite clearly :)
[23:28:21] <Marc Worrell> luckily not so much for us to merge.
[23:28:35] <Maas95625> No, indeed.
[23:28:46] <Marc Worrell> (nothing that I can see, maybe later that content-md5 support)
[23:28:50] <Maas95625> Strangeness too.. https://github.com/basho/webmachine/commit/1ab67436e0b63f44d828cf36f8289ddb2829d5a0
[23:29:15] <Maas95625> proplists ftw!
[23:29:42] <Marc Worrell> aha, keytake/keydelete combo
[23:30:02] <Marc Worrell> (but what does it matter, those lists are so short...)
[23:30:15] <Maas95625> hehe..
[23:30:32] <Marc Worrell> just the options for mochiweb...
[23:30:40] <Maas95625> still you think it is very expensive.
[23:30:52] <Marc Worrell> me?
[23:31:00] <Maas95625> no programmers
[23:31:04] <Maas95625> lots of them
[23:31:10] <Maas95625> other i mean
[23:31:12] <Marc Worrell> LOL
[23:31:42] <Marc Worrell> yep - still need to remove a dict from webzmachine - is used for a couple of items, proplists will be easier&faster
[23:32:29] <Maas95625> on the list with the thing...
[23:36:59] <Marc Worrell> https://github.com/basho/webmachine/pull/104
[23:38:20] <Maas95625> Nice.. :-)
[23:38:37] <Maas95625> didn't know orddict was a proplist.
[23:39:21] <Maas95625> keeps the stuff sorted though.
[23:39:43] <Marc Worrell> this one we can use :)
[23:39:46] <Marc Worrell> https://github.com/basho/webmachine/commit/ff54bd3cda8bbf65393204adc271441e0fa00230
[23:40:44] <Marc Worrell> will help with the elli move
[23:41:03] <Maas95625> So you see more clearly what is what
[23:44:02] <Marc Worrell> yep, merging by hand
[23:44:44] <Maas95625> Looking at elli right now...
[23:46:41] <Maas95625> No documentation of course, but we are used to that.
[23:51:48] <Maas95625> You have to supply a handler which then get events like: request complete, chunk_complete. It may even be easier to interface it with webmachine than mochiweb.
[23:54:05] <Maas95625> Looks like elli misses https though.
[23:55:23] <Maas95625> But that difference is almost invisible for the lib itself.
[23:55:25] Maas95625 leaves the room
[23:58:08] Maas joins the room
Powered by ejabberd Powered by Erlang Valid XHTML 1.0 Transitional Valid CSS!