Tuesday, 5 March 2013< ^ >
arjan has set the subject to: Zotonic - the Erlang Content Management Framework
Room Configuration

[00:08:59] Arjan leaves the room
[00:26:49] jeff.5nines joins the room
[00:36:00] Maas leaves the room
[02:46:26] jeff.5nines leaves the room
[02:47:17] jeff.5nines joins the room
[03:31:09] jeff.5nines leaves the room
[03:32:20] jeff.5nines joins the room
[05:38:32] aRyo joins the room
[05:39:27] aRyo leaves the room
[08:39:30] Arjan joins the room
[08:54:54] Andreas Stenius joins the room
[10:02:34] Maas joins the room
[10:04:09] <Maas> O no, have to go back to the store with new mbp. Sigh...
[10:05:42] <Maas> The keyboard is a bit wonky...
[10:09:02] <Maas> Can't work in the office,
[10:09:12] <Maas> can't work at home
[10:09:24] <Maas> pff
[10:11:26] Maas leaves the room
[10:12:07] <Andreas Stenius> wonky... I thought the fruit company had quality products.. :p
[10:16:47] <Andreas Stenius> Arjan, Marc Worrell: what do you think of pull #515 ?
(I'm writing a reply to Greg..)
[10:53:32] Maas joins the room
[10:54:25] <Maas> Keyboard fixed.. A bit of leftover plastic... geeeez
[10:54:45] <Andreas Stenius> heh
[11:39:52] Maas leaves the room
[12:18:41] <Arjan> sorry andreas, I am really busy
[12:18:54] <Arjan> but you're doing a good job merging things! :D
[12:19:06] <Arjan> sorry I can't help or review these days
[12:22:31] <Andreas Stenius> ok, np. I'll do my best, hope I don't mess it up too bad :D
[13:16:27] <Marc Worrell> How much overlap is there between Arjan's OTPification branch and Amiramix work?
[13:16:45] <Andreas Stenius> I think arjan has worked more with the modules side of things..
[13:16:56] <Andreas Stenius> which Grzegorz hasn't touched
[13:17:14] <Andreas Stenius> fingertwisting name, that :p
[13:28:43] <Arjan> my rebar branch?
[13:28:59] <Arjan> basically, all I did was creating a script to make each zotonic module into an OTP app
[13:29:03] <Arjan> and adding a rebar.config
[13:29:14] <Arjan> I think they are pretty orthogonal
[13:29:18] <Arjan> I'd start with his branch
[13:29:24] <Arjan> and then do mine on top of that at one point
[13:29:29] Maas joins the room
[13:29:33] <Andreas Stenius> sounds good :)
[13:30:05] <Andreas Stenius> I like rebar's ability to keep all subprojects deps at a toplevel deps dir
[13:31:03] <Maas> That is really nice indeed. Also good for the db layer.
[13:31:56] <Marc Worrell> yes, also good for modules - when they become rebar'ized
[13:32:15] <Marc Worrell> @arjan sounds good
[13:33:00] <Andreas Stenius> can we break out those parts in deps/ that currently live in the zotonic repo to their own instead? or have them else where so they're not mixed up with remote deps?
[13:33:54] <Andreas Stenius> It's a strain on the head to keep track of what is dep that is managed by submodule/rebar and what lives in the zotonic repo if they both go into deps/
[13:35:09] <Arjan> I agree
[13:35:18] <Arjan> lets put our mochiweb in zotonic/mochiweb on github
[13:35:25] <Arjan> I think that's the only one
[13:35:36] <Arjan> or not?
[13:35:40] <Maas> Indeed was just checking...
[13:35:44] <Andreas Stenius> iconv too
[13:35:56] <Maas> That one is gone
[13:35:57] <Andreas Stenius> but then we have eiconv now... or?
[13:36:10] <Andreas Stenius> then it's just mochiweb
[13:36:20] <Andreas Stenius> +1 for zotonic/mochiweb
[13:37:13] <Andreas Stenius> I can't create repos for zotonic... :p
[13:38:12] <Marc Worrell> yes, clone mochiweb - we need to check that one anyway for R16
[13:39:10] <Marc Worrell> @kaos - now you can ;-)
[13:39:13] <Maas> Pff yes. Instead of removing the parameterized module it may be nicer to throw in cowboy or elli
[13:39:18] <Andreas Stenius> :D
[13:39:42] <Marc Worrell> @maas - indeed and make those lists binaries
[13:39:54] <Andreas Stenius> well that's a fun project (not just joking) :p
[13:40:05] <Maas> Two birds with one stone.
[13:40:17] <Marc Worrell> :-) they should both be done - so why not combine it
[13:40:31] <Marc Worrell> will take some work from everybody - but is the direction to go anyway
[13:40:39] <Marc Worrell> all data from outside should be binary
[13:40:51] <Marc Worrell> (unless in the middle of some processing)
[13:41:24] <Andreas Stenius> Marc Worrell: how do you mean, combine it, exactly? (just want to avoid misunderstandings) :)
[13:41:42] <Marc Worrell> yes - new library and binaries
[13:41:55] <Maas> The url and html parsing of mochweb is still nice though, although recent otp can do url parsing too
[13:42:13] <Marc Worrell> we can force R14+ — no problem
[13:42:40] <Andreas Stenius> I'll move mochiweb first. Leaving the cowboy/elli replacement as a future exercise
[13:42:48] <Marc Worrell> mochiweb parsing can be used - they have some binary interfaces as well
[13:43:03] <Marc Worrell> yes - move it first ;-)
[13:43:58] <Maas> It is quite fast too and very stable too. It can take quite a bit of malformed html.
[13:51:46] <Marc Worrell> yes - we must keep that one
[13:53:03] <Marc Worrell> elli doesn't seem to have websocket handlers - that is a nice thing of cowboy
[13:53:13] <Marc Worrell> Though I don't like cowboy's API so much
[13:53:19] <Andreas Stenius> which shouldn't be too hard to steal :p
[13:53:33] <Marc Worrell> everything returns a new cowboy record, even when nothing changes
[13:53:38] <Andreas Stenius> I like elli, as it seems like it's really easy to plug new stuff into it
[13:53:43] <Maas> I like elli too...
[13:54:01] <Marc Worrell> and it is simple - what we need underneath webzmachine
[13:54:04] <Maas> Zotonic now implements ws anyway.
[13:54:23] <Maas> Just http.. and nothing else.
[13:54:38] <Maas> thank you very
[13:54:43] <Maas> much :-)
[13:54:52] <Marc Worrell> yes, the upgrade should be done how we do it - not some header sniffing by the library
[13:55:02] <Arjan> if you are splitting of mochiweb, please keep the history
[13:55:20] <Marc Worrell> yep :)
[13:55:38] <Arjan> git --filter-branch wtf :)
[13:56:40] <Marc Worrell> I like Elli too - good story :)
[13:56:47] <Marc Worrell> http://www.wooga.com/2012/08/woogatech_elli/
[13:56:54] <Marc Worrell> and instrumentation
[13:57:49] <Andreas Stenius> ah, right. that's a better way to do it :) ok, I'll redo it :p
[13:59:14] <Marc Worrell> wtf? :p are you cursing here?
[13:59:38] <Andreas Stenius> indeed I think he is :p
[14:00:23] <Andreas Stenius> but quite rightly so, I think no sane person keeps the args for filter-branch on-top of their head ;)
[14:00:25] <Marc Worrell> was it the g** or the w** word that is worst? :p
[14:00:43] <Andreas Stenius> heh :p
[14:01:11] <Andreas Stenius> nah, I'm quite fond of git at the moment... let's ask that again in 15 minutes...
[14:02:19] <Maas> haha, i'm glad the git submodules magic is soon a thing of the past :-)
[14:02:56] <Maas> replaced with rebar magic...
[14:04:32] <Maas> On the other hand, tying zotonic to the master branch of all its deps makes zotonic master a bit more bleeding edge....
[14:04:40] <Andreas Stenius> no, I'm in love with git...! `git filter-branch --subdirectory-filter deps/mochiweb -- --all` ! :D
[14:05:03] <Maas> what are you talking about :-D
[14:05:44] <Marc Worrell> I think his kid is sitting on the keyboard
[14:06:06] <Andreas Stenius> lol
[14:06:09] <Marc Worrell> or he is high - what are those swedes drinking?
[14:06:11] <Maas> git kid
[14:06:47] <Andreas Stenius> voila, the history is there! :D
[14:06:53] <Andreas Stenius> https://github.com/zotonic/mochiweb
[14:07:45] <Andreas Stenius> even the tags :)
[14:08:09] <Marc Worrell> git history and tags to voila —justdoit —yes —really
[14:10:11] <Andreas Stenius> now, how do I get my working dir back in order... :p ?
[14:11:56] <Andreas Stenius> ah, ok. I tried with a bunch of git reset's but it didn't wanna do it, but a git pull upstream master did it... finally ;)
[14:12:07] <Arjan> :)
[14:12:09] <Arjan> cool
[14:12:22] <Arjan> maybe good to change the README for our mochiweb
[14:12:30] <Arjan> as it is not a real fork anymore
[14:12:31] <Andreas Stenius> not a bad idea
[14:12:50] <Maas> As long as git doesn't pull you downstream everything is ok ;-)
[14:14:04] <Maas> Mochiweb is changed quite a bit.
[14:14:40] <Andreas Stenius> those changes (37, to be precise) are easier to spot now, too :)
[14:14:54] <Maas> Nice :-)
[14:14:56] <Andreas Stenius> well, 37 since the move to hg, any how
[14:15:10] <Andreas Stenius> as the subversion history wasn't kept :/
[14:16:11] <Arjan> z_html:unescape("&#231;").
[14:16:17] <Arjan> that does not seem right, does it?
[14:16:47] <Marc Worrell> i think the unescape did only do what it knows :p
[14:17:00] <Maas> There is probably a story in there somewhere :-)
[14:17:05] <Marc Worrell> should do more, I think
[14:17:27] <Andreas Stenius> http://en.wikipedia.org/wiki/%C3%87
[14:18:12] <Arjan> yes
[14:18:39] <Andreas Stenius> so it should just spit out an utf char, right..
[14:19:19] <Arjan> it should
[14:19:23] <Andreas Stenius> uhm, or unicode.. bah, all those charsets and encodings... I never really got into it
[14:19:46] <Arjan> a binary utf-8 encoded string, I'd go for
[14:19:55] <Arjan> xmerl for instance uses lists for strings
[14:20:01] <Arjan> but those lists are not utf-8, but unicode
[14:20:13] <Arjan> luckily there is unicode:characters_to_binary/1
[14:20:18] <Arjan> which does utf8 encoding
[14:23:27] <Maas> mochiweb_html also knows its way around xml :-) It expects utf-8 input
[14:23:46] <Maas> And spits out a tree with binaries in it.
[14:29:01] <Arjan> ok
[14:29:21] <Andreas Stenius> mochiweb move completed
[14:29:31] <Arjan> but none of you know of a more complete way to decode HTML entities?
[14:29:36] <Arjan> Andreas Stenius: nice!!
[14:35:41] <Maas> @arjan, I'll have to check that. mochiweb converts most of them with tokenize_charref
[14:36:32] <Maas> There is a big list in mochiweb_charref
[14:37:04] <Maas> Or maybe erlang needs an expat nif
[14:37:10] <Maas> :-p
[14:39:23] <Maas> xml is a three letter word too
[14:39:28] <Arjan> hmm
[14:41:25] <Maas> Maybe it misses an entity?
[14:47:38] Marc Worrell leaves the room: Disconnected: Replaced by new connection
[14:47:38] Marc Worrell joins the room
[14:49:30] <Maas> Working with vagrant is nice, but the shared directory between virtualbox and the host is slow and causes timeouts...
[14:49:34] <Maas> sometimes.
[14:49:54] <Maas> 13:47:25.399 [error] gen_server 'z_template$sitewww' terminated with reason: {timeout,{gen_server,call,['z_module_indexer$sitewww',{find_all,template,"_html_head.tpl",desktop}]}}
[14:53:55] <Maas> It is probably busy scanning at that time.. maybe that should be done in the background so lookups are not blocked by it.
[15:15:15] <Arjan> implementing a proper z_html:unescape
[15:36:09] <Maas> btw: > mochiweb_html:parse(<<"<tag>&#231;</tag>>>)).
[15:57:15] <Arjan> and then not depending on mochiweb
[16:37:41] Arjan leaves the room
[16:40:02] Arjan joins the room
[17:29:12] Arjan leaves the room
[17:31:45] Maas leaves the room
[19:51:00] Arjan joins the room
[19:53:09] <jeff.5nines> Hi Arjan, I found out some stuff about my webfaction account
[19:54:51] <jeff.5nines> Apparently webfaction blocks any call from the localhost to the public IP address. I found a work around here. http://community.webfaction.com/questions/2366/can-i-use-rabbit-mq-on-the-shared-servers
[19:57:18] <jeff.5nines> The downside is I need to start erlang something like this: erl -sname jbell -kernel inetrc '"/home/username/.erl_inetrc"'
[19:58:33] <jeff.5nines> I can override the zotonic scripts manually but I'm not sure what a good long term solution will be if I'm using a git version
[19:58:36] <Andreas Stenius> I think you can run the erlang node bound to you localhost, and still have zotonic listen on the public ip
[19:59:12] <jeff.5nines> Hey Andreas, I say your status was not available :)
[19:59:31] <Andreas Stenius> I was away putting my youngest to bed.. ;)
[20:00:22] <jeff.5nines> So the erland docs say ...
[20:00:28] <jeff.5nines> Note that the kernel variable inetrc overrides this environment variable.If no user configuration file is specified and Erlang is started in non-distributed or short name distributed mode, Erlang will use default configuration settings and a native lookup method that should work correctly under most circumstances. Erlang will not read any information from system inet configuration files (like /etc/host.conf, /etc/nsswitch.conf, etc) in these modes, except for /etc/resolv.conf and /etc/hosts that is read and monitored for changes on Unix platforms for the internal DNS client inet_res <http://erlang.org/doc/man/inet_res.html>.
[20:00:35] <Andreas Stenius> I and Maas considered that it might've been webfaction that had something to do with your issues.. sort of glad it was ;)
[20:02:03] <Andreas Stenius> I was thinking that it ought to listen to the IP for the hostname used in the node name, as in -sname test@localhost
[20:07:12] <jeff.5nines> yeah, that worked for my shell sessions. Now to see how to force zotonic script to use localhost
[20:09:14] <Andreas Stenius> ZOTONIC_DISTRIBUTED=false HOSTNAME=localhost
[20:12:57] <jeff.5nines> Whooot, whoot!
[20:13:23] <Andreas Stenius> sounds like it's working :D
[20:17:44] <jeff.5nines> Chink in the armor Ted. Something's not right with the site now.
[20:20:38] <jeff.5nines> Okay, now we're good. Nothing a little z:m() and z:restart() won't fix right.
[20:21:38] <Arjan> :)
[20:24:42] <jeff.5nines> Thanks for everyone's help and input. I actually learned quit a bit :)
[20:25:29] <jeff.5nines> /quit/quite
[20:25:59] <Andreas Stenius> yeah, it's when you face issues you actually learn how stuff works. Everything that just works you don't need to pop the hood on ;)
[20:59:31] Maas joins the room
[21:15:18] <jeff.5nines> For what it's worth, now that I've figured out the localhost connection issue, deploying on webfaction is quite easy. I really like Andreas' idea of a hosting service though.
[21:16:08] <Andreas Stenius> Yeah, I think there's quite a number of potential zotonic users out there that doesn't want to bother with servers and stuff..
[21:17:29] <Maas> I say, lean startup :-) Simple site with a signup on it. Just to see how much interest there is.
[21:19:06] <Maas> You need more than a handful of people I think.
[21:28:13] Arjan leaves the room
[21:37:52] <Maas> Toying around with the vm... Btw, if you create the database like this: # CREATE DATABASE zotonic WITH OWNER zotonic ENCODING = 'UTF-8'
LC_CTYPE = 'en_US.utf8' LC_COLLATE = 'en_US.utf8'
TEMPLATE template0;
[21:38:10] <Maas> The encoding of the current terminal does not matter anymore
[21:38:40] <Andreas Stenius> good to know :)
[21:38:45] <Maas> template0 also allows you to skip the \c zotonic
[21:38:55] <Maas> and # CREATE LANGUAGE "plpgsql";
[21:39:11] <Andreas Stenius> about the hosting, yeah, planning to start lean..
[21:39:39] <Andreas Stenius> the create lang hasn't been needed since version 9, as it is created per default...
[21:40:16] <Maas> :-) Didn't know that one... The production sites or on postgres 8.something.
[21:41:45] <Maas> Busy setting up zotonic master
[21:42:33] <Maas> The vm setup is nice. The only issue I have is timeouts when the module indexer is scanning and another part of the system needs something....
[21:43:48] <Maas> When that happens zotonic will restart the site. It can even lead to a restart loop
[21:44:12] <Maas> scanning.... timeout... scanning... kind of goofy
[21:45:00] <Maas> The scanning should be a separate process so that the indexer will still answer queries with the old values I guess.
[21:45:51] <Andreas Stenius> yeah, that sounds like a good idea
[21:46:52] <Maas> The reason for this that this surfaces with the vm setup is that the shared folder between the host and the vm where zotonic is sitting is very slow..
[21:47:03] <Maas> I've seen this happening on the atom too.
[21:49:03] Arjan joins the room
[21:53:51] <Maas> Did something change in the boot recently? This when starting zotonic master.. {"init terminating in do_boot",{{badmatch,{error,{{function_clause,[{filename,join,[{error,bad_name},"config"]},{filename,join,1},{z_config,ensure_config,0},{z_config,get_dirty,1},{zotonic_sup,init,1},{supervisor,init,1},{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]},{zotonic_app,start,[normal,[]]}}}},[{zotonic,start,1},{init,start_it,1},{init,start_em,1}]}}
[21:54:56] <Andreas Stenius> could be that "config" should be config (i.e. an atom)
[21:55:17] <Maas> filename:join can do that/
[21:55:39] <Andreas Stenius> yeah, it does... hmm...
[21:55:42] <Maas> > filename:join([atom]).
[21:56:06] <Maas> looks like it tries to join an error bad_name with config.
[21:56:15] <Andreas Stenius> uh, wait, isn't the {error, bad_name} passed to filename:join
[21:56:22] <Andreas Stenius> precisely
[21:57:21] <Maas> z_utils:lib_dir(priv) returns an error... which is happily used later on.
[21:58:01] <Andreas Stenius> hah, I was suspecting the z_utils:lib_dir change.. hence my no, not string, atom only comment...
[21:58:19] <Maas> yup. > z_utils:lib_dir(priv).
[21:58:20] <Andreas Stenius> but how did we end up there?
[21:58:34] <Andreas Stenius> what? no, that feels wrong..
[21:58:34] <Maas> filename:join([z_utils:lib_dir(priv), "config"]).
[21:58:37] Arjan leaves the room
[21:58:40] <Andreas Stenius> hmmm
[21:59:49] <Andreas Stenius> well, I merged a pull request from amiramix that changed this..
[22:00:00] <Maas> :-) Little details.
[22:00:08] <Andreas Stenius> lets see
[22:01:22] <Andreas Stenius> (zotonic001@morpheus)2> code:lib_dir(zotonic, priv).
(zotonic001@morpheus)3> code:lib_dir(zotonic).
[22:01:26] <Andreas Stenius> what do you get on those?
[22:01:29] <Maas> code:lib_dir(mnesia).
[22:01:39] <Maas> code:lib_dir(zotonic).
[22:01:51] <Maas> zotonic is a good name ;-)
[22:01:57] <Andreas Stenius> lol-
[22:02:28] <Maas> This is a vanilla R14B04 on ubuntu
[22:03:30] <Maas> So somehow it doesn't know about the zotonic application.
[22:03:48] <Andreas Stenius> well, isn't that strange. also, it seems to work for me..
[22:05:30] <Marc Worrell> I also just updated the master here - recompiled and seems to work fine for me
[22:05:41] <Maas> Nice one.
[22:05:48] <Andreas Stenius> phew :)
[22:06:01] <Marc Worrell> (zotonic009@miffy)3> code:lib_dir(zotonic, priv).
[22:06:02] <Andreas Stenius> ok, Maas, seems like a config/setup issue on your box..
[22:06:33] <Marc Worrell> (zotonic001@Lamma)4> code:lib_dir(zotonic, priv).
[22:07:08] <Andreas Stenius> just pulled latest, rebuild and tested, still seems ok... hmmm
[22:07:22] <Andreas Stenius> have you named your zotonic dir something other than zotonic perhaps?
[22:07:31] <Maas> No
[22:07:37] <Andreas Stenius> ok
[22:07:46] <Maas> Checking another vm
[22:10:52] <Maas> Crazy. > application:start(zotonic).
(zotonic001@zotonic-dev)3> code:lib_dir(zotonic).
[22:11:38] <Andreas Stenius> weird
[22:12:24] <Maas> Erlang R14B04 (erts-5.8.5) [source] [rq:1] [async-threads:0] [kernel-poll:false]
[22:19:10] <Andreas Stenius> I was on R15B01
[22:19:31] <Andreas Stenius> ok, bedtime over here...
[22:19:47] Andreas Stenius leaves the room
[22:19:50] <Maas> Here on another R14B04 box (not a vm).. code:lib_dir(zotonic).
[22:31:26] <Maas> Running in the vm, then the directory zotonic is sitting is is /vagrant
[22:33:03] <Maas> Then code:lib_dir(zotonic) doesn't work. On the other ubuntu box I've named zotonic differently too..
[22:36:50] <Maas> Of course simple renaming also does not work because of git magic...
[22:44:35] <Maas> So the directory zotonic is living in MUST be called zotonic.
[22:44:54] <Maas> or zotonic-Vsn ...
[22:47:28] <Maas> Makes sense if zotonic is embedded in another app, but not when it is running stand alone.
Powered by ejabberd Powered by Erlang Valid XHTML 1.0 Transitional Valid CSS!