Thursday, 14 March 2013< ^ >
arjan has set the subject to: Zotonic - the Erlang Content Management Framework
Room Configuration

[01:10:24] vladimir.chujkov joins the room
[06:43:05] vladimir.chujkov leaves the room
[06:43:57] vladimir.chujkov joins the room
[08:09:58] vladimir.chujkov leaves the room
[08:10:36] vladimir.chujkov joins the room
[08:24:21] Arjan joins the room
[08:43:23] vladimir.chujkov leaves the room
[08:45:45] Andreas Stenius joins the room
[09:03:26] vladimir.chujkov joins the room
[09:13:27] vladimir.chujkov leaves the room
[09:16:13] Maas joins the room
[09:17:46] <Maas> Things are busy on the github repo these days...
[09:20:17] <Marc Worrell> I was discussing some mailinglist stats with Arjan the other day.
[09:20:55] <Marc Worrell> Would it be possible to add arbitrary time based stats (like in …. per hour over whatever period) ?
[09:21:24] <Marc Worrell> In this case follow ups, or openings of e-mails by recipients.
[09:21:25] <Maas> Aha, that could probably be merged with the stats andreas and me made
[09:21:40] <Marc Worrell> I was thinking so, so I didn't want to do anything :p
[09:22:40] <Marc Worrell> We would like better feedback for the mailings, like amount bounced (and which ones, why, how often etc), amount queued, amount opened, amount followed-up etc
[09:22:44] <Maas> For collection of the stats we use folsom. It has quite a number of ways to count and measure things.
[09:23:03] <Marc Worrell> Will be great to have such stats - especially the live ones :-)
[09:24:47] <Maas> We are looking into merging it with master soon. One thing we need to check is if the histogram collection of the durations isn't causing a memory leak. It collects things into different bins on ets. It is not clear to me if it leaks or not.
[09:25:27] <Marc Worrell> hmmm, indeed, better test
[09:25:41] <Maas> Then integration with zotonic makes it relatively easy to add custom counters.
[09:26:07] <Maas> There are a couple of different histogram implementations inside. I just picked one.
[09:26:46] <Maas> I chose this one. https://github.com/boundary/folsom#exdec
[09:27:39] <Maas> Maybe this one is better. https://github.com/boundary/folsom#slide
[09:28:13] <Marc Worrell> would it be possible to start some new stats, and the after a period that folsom cleans up that stats when it isn't used anymore?
[09:28:38] <Marc Worrell> that would be great when sending mailing lists, dynamic stats
[09:29:39] <Maas> Yes you can start and stop individual counters I believe.
[09:30:28] <Andreas Stenius> yep, and the stats page is dynamic, so that new stats can be added on the fly (as well as remove old stats no longer used)
[09:31:21] <Andreas Stenius> but I'd prefer to merge this soonish, that way it's easier to test additional stats for different parts of zotonic.. ;)
[09:32:15] <Marc Worrell> yes - merge it soon - then we can just start testing etc etc
[09:32:16] <Andreas Stenius> the chart stuff is pretty much proof of concept still...
[09:32:29] <Marc Worrell> only by really using it we can discover problems
[09:32:35] <Maas> :-) yup. I'm awefully busy right now.
[09:32:55] <Andreas Stenius> yep, it's using it that'll lead the way for how to best implement it.. :)
[09:32:55] <Marc Worrell> I'm also busy meeting a deadline for a MaxClass clone...
[09:32:57] <Maas> Should I just do a pull request?
[09:33:19] <Marc Worrell> yep - we can all do the merge :p
[09:33:44] <Maas> Need to fix some things for the new build stuff I think. And merge one change in webzmachine.
[09:33:49] <Andreas Stenius> pull's let you get a pretty clear picture of what the merge will look like.. :)
[09:34:14] <Maas> And adding deps is so easy now...
[09:34:20] <Andreas Stenius> indeed :)
[09:34:34] <Maas> Here it comes. :-)
[09:37:26] <Andreas Stenius> hah, cool.. it included all or previous commit comments along in the pull request.. :p
[09:38:32] <Maas> Is that nice or?
[09:38:37] <Andreas Stenius> yep
[09:39:05] <Andreas Stenius> I can make another pull request to you z_stats branch to fix the conflicts, if you want.. ?
[09:39:46] <Marc Worrell> In other news:
[09:39:47] <Marc Worrell> Now, instead, Google wants users to rely on the Google Calendar APIs, according to today’s blog post <http://googleblog.blogspot.com/2013/03/a-second-spring-of-cleaning.html>:
CalDAV API will become available for whitelisted developers, and will be shut down for other developers on September 16, 2013. Most developers’ use cases are handled well by Google Calendar API, which we recommend using instead.
[09:39:49] <Maas> Sure, I don't have the time right now.
[09:40:27] <Andreas Stenius> I might not get to it until tonight either..
[09:40:31] <Maas> They are shutting down quite a bit of stuff. Twitter tl was filled.
[09:41:18] <Maas> :-) That is how things go. Finally have some time to finish the bus fallback mechanism.
[09:41:29] <Marc Worrell> Yep, but this CalDAV thing is pretty nasty. They just shut down Active Sync and Microsoft was busy implementing CalDAV as an alternative.
[09:42:15] <Marc Worrell> I guess we better not use Google for our calendars and other data - soon we can't get it out of them anymore.
[09:42:31] <Maas> Is this the new battle ground? Shutting down standardized api's.
[09:42:54] <Marc Worrell> looks like it - just like Twitter
[09:43:03] <Andreas Stenius> indeed, I'm usually reluctant to rely on external parties that hasn't an open way out regardless of their business desicions..
[09:43:11] <Marc Worrell> they are all closing up - nice opportunity for open software :)
[09:43:12] <Maas> For calendars, notes.
[09:44:48] <Maas> That is right. Making a better ui than google is easy too. :-)
[09:46:10] vladimir.chujkov joins the room
[09:53:05] <Maas> Looking into updating jquery and jquery ui. Do we use a custom rolled jquery ui? And do we use any of the standard css and images for ui?
[09:53:42] <Andreas Stenius> I think it is a stock download of both... but I'm not sure ;)
[09:54:18] <Maas> Hmm... the stock download contains css and images (for ui)
[09:54:57] <Andreas Stenius> there are some ui images here: https://github.com/zotonic/zotonic/tree/master/modules/mod_base/lib/images
[09:55:15] <Andreas Stenius> and css: https://github.com/zotonic/zotonic/tree/master/modules/mod_base/lib/css
[09:55:20] <Maas> Oh, those ui icons.
[09:56:01] <Maas> Ignoring those then :-p
[09:57:12] <Andreas Stenius> was thinking that it could be an idea to break out jquery specifics from mod_base into a mod_jquery module... including the js, css, actions, tags etc..
[09:57:41] <Andreas Stenius> that way it will be clearer what features are tied to jQuery, specifically..
[09:58:01] <Maas> Was thinking that too. Using jquery ui for mobile is also not that nice.
[09:58:05] <Andreas Stenius> and easier to replace with another js library (for anyone that wants to do that... :p )
[10:02:17] <Marc Worrell> The $(…) syntax is in a lot of places
[10:02:50] <Marc Worrell> I am not sure you want to build on something else than jquery - can be a lot of pain when you need to do some specific things
[10:03:06] <Andreas Stenius> yeah, but that could be abstracted out I think.. or put the responsibility of supporting that syntax on the replacing module..
[10:03:48] <Andreas Stenius> indeed. the major drive for a split is to have a clear separation, not swappability of jquery itself
[10:04:16] <Andreas Stenius> I have no intentions of replacing jquery :p
[10:08:54] arcusfelis joins the room
[10:13:23] <Arjan> is stats ready for merge?
[10:13:42] <Andreas Stenius> needs conflict resolution..
[10:13:58] <Andreas Stenius> and maas wanted to check that there's not a mem leak in folsom..
[10:13:59] <Maas> :-) and time to do the conflict resolution.
[10:14:17] <Arjan> many conflicts?
[10:14:29] <Andreas Stenius> mainly the swap to rebar deps
[10:14:37] <Maas> new build deps indeed.
[10:14:48] <Arjan> oh that
[10:14:49] <Andreas Stenius> also, webzmachine needs to be merged first, I think
[10:14:50] <Maas> and need to cherry pick an update to webzmachine
[10:15:08] <Arjan> ok
[10:15:41] <Maas> later this evening....
[10:15:56] <Andreas Stenius> yep, I'll work on getting the pull clean
[10:16:28] <Andreas Stenius> also later tonight..
[10:17:19] <Maas> Ok, then I'll do the cherry pick and look into folsom stat collection.. We wanted to merge soon. During development I've already found it useful without all the graphics.
[10:18:00] <Andreas Stenius> indeed, and for the graphics, I could use more use case feedback to guide the implementation and design :)
[10:18:51] <Maas> I needs to be used in practice indeed.
[10:24:11] <Maas> Btw @arjan, how did you download jquery ui for zotonic. All the features enabled and then "no theme"
[10:25:38] <Arjan> yes, I think so
[10:26:08] <Maas> (Updating jquery and jquery ui)
[11:03:25] <Arjan> https://gist.github.com/arjan/5160175
[11:03:29] <Arjan> pretty big change log
[11:04:40] <Arjan> maas,
[11:04:48] <Arjan> why didnt you go directly to jquery 1.9.1?
[11:04:59] <Arjan> does it have breaking changes?
[11:05:26] <Andreas Stenius> nice change log :)
[11:05:42] <Andreas Stenius> the tpl mode still needs more work btw..
[11:05:56] <Andreas Stenius> I noticed that while working on the stats template.. ;)
[11:06:38] <Andreas Stenius> but it didn't crash on me, always something :p
[11:07:01] <Maas> 1.9 deprecates some interfaces.
[11:07:13] <Maas> (removed the deprecated ones)
[11:07:37] <Marc Worrell> and two more changes
[11:07:58] <Maas> Also .bind is not the preferred way to do stuff.
[11:08:22] <Maas> We should probably check the generated js for that.
[11:08:27] <Arjan> jup
[11:08:59] <Arjan> Marc Worrell: I'm planning to do the 0.9.1 release this afternoon, around 15:00
[11:09:11] <Arjan> so you still have some time ;)
[11:09:20] <Marc Worrell> I pushed them :p
[11:09:27] <Marc Worrell> see your e-mail
[11:09:31] <Marc Worrell> just two fixes
[11:09:39] <Maas> The new way (.on) is handy for dynamically added things. Probably better for postback forms.
[11:16:31] <Arjan> the baby is awake so no release just yet :)
[11:38:14] Maas leaves the room
[11:52:19] jeff.5nines joins the room
[12:08:25] Maas joins the room
[12:09:39] Arjan leaves the room
[12:10:23] Arjan joins the room
[12:33:01] Maas leaves the room
[12:33:17] Maas joins the room
[12:33:22] Maas leaves the room
[13:11:49] <Andreas Stenius> hmm.. github.com under DDos attack...
[13:12:43] Maas joins the room
[13:13:06] <Maas> Again, happening very often these days.
[13:21:54] <Maas> At work we use http://www.fogcreek.com/kiln/. It was mercurial only now you can use git and hg on the same repository.
[13:23:29] <Maas> It is a paid for service though.
[13:28:09] <Maas> Getting a bit worried about elli. Made the pull request last monday, but no reaction whatsoever so far.
[13:36:22] <Andreas Stenius> hmm... maybe he's out of town... ?
[13:37:00] <Maas> Could be. I'll ping him to tell about our plan.
[13:38:54] <Andreas Stenius> we all agree we prefer the elli design over cowboy... but perhaps we should also consider the community supporting the lib.. in that perspective, cowboy has a rather big advantage..
[13:39:19] <Maas> Yes, that will be one of my questions.. :-)
[13:40:08] <Maas> I guess we could also attract the community with a webzmachine + elli combo. But that is more work for us.
[13:41:16] <Andreas Stenius> indeed. I was considering that we could take on elli development.. but as you say, that'll be more work for us..
[13:46:45] <Maas> webzmachine + cowboy would probably even attract more people.
[13:47:02] <Andreas Stenius> what was the show stoppers for cowboy?
[13:47:10] <Andreas Stenius> I remember you had at least two issues
[13:47:21] <Maas> Well show stoppers... that is a bit harsh.
[13:47:29] <Andreas Stenius> heh
[13:47:42] <Andreas Stenius> well, ok, I guess they're fixable ?
[13:48:00] <Maas> They use all small caps headers which don't play nice with other stuff.
[13:48:38] <Andreas Stenius> that's an easy search-replace, isn't it?
[13:48:48] <Maas> I remember that @marc already pinged the author about that.
[13:49:04] <Maas> He doesn't think it is a problem.
[13:49:25] <Andreas Stenius> and then marc mentioned something about copying the context even if it wasn't changed (or something like that...)?
[13:49:44] <Maas> So then we still have our separate branch.
[13:57:01] <Andreas Stenius> with elli, we'll have more work to do, but we can do it the way we want it. With cowboy, alot of stuff is already there, but perhaps not always implemented the way we like it..
[13:59:48] <Maas> There is a bit of overlap between what cowboy does and webmachine. We just need the http. But we can always branch of and make cowzboy if needed ;-)
[14:09:21] <Maas> We will see. We can still decide which way to go. There are basically three routes.
[14:11:46] <Maas> 1 - change mochiweb (or wait until it is changed by somebody else)
[14:12:06] <Maas> 2 marry cowboy to webzmachine
[14:12:20] <Maas> 3 - or elli
[14:12:28] <Maas> 4 - something else?
[14:13:58] <Maas> oh, and there is also misultin
[14:20:12] <Maas> Nice.. got an email back from him immediately
[15:06:40] Maas leaves the room
[15:57:56] Maas joins the room
[16:03:38] arcusfelis leaves the room
[16:05:12] <Andreas Stenius> what needs to be fixed in mochiweb, then? is it "only" to convert from lists to binaries? or is it more.. ?
[16:07:56] arcusfelis joins the room
[16:12:57] <Arjan> email back from who?
[16:18:19] Andreas Stenius leaves the room
[16:21:44] Andreas Stenius joins the room
[16:25:12] Maas leaves the room
[16:36:31] Maas joins the room
[17:09:29] jeff.5nines leaves the room
[17:09:44] jeff.5nines joins the room
[17:10:48] Maas leaves the room
[17:16:31] arcusfelis leaves the room
[17:21:03] Maas joins the room
[17:21:49] Maas leaves the room
[17:24:30] arcusfelis joins the room
[19:55:19] Maas joins the room
[19:57:58] <Arjan> 0.9.1 is out the door
[20:00:28] <Marc Worrell> yeah!
[20:00:45] <Arjan> now the debian PPA
[20:01:04] <Maas> Kaboom baby. Very nice... :-)
[20:01:17] <Marc Worrell> :)
[20:15:07] <Andreas Stenius> cool! :D
[20:23:24] <Marc Worrell> uh oh - Maas,je build failed
[20:23:30] <Marc Worrell> :p
[20:23:36] <Maas> huh?
[20:23:58] <Maas> busy with the stats.
[20:24:20] <Marc Worrell> oh - GitHub probleem denk ik
[20:24:21] <Marc Worrell> Using worker: bluebox-linux-1.worker.travis-ci.org:travis-linux-19
$ export DB=postgres
$ git clone --depth=100 --quiet --branch=master git://github.com/zotonic/zotonic.git zotonic/zotonic
cd zotonic/zotonic
$ git checkout -qf 19ebc178c75a6836e606c424ec987f684140e900
$ source $HOME/otp/R15B01/activate
$ git submodule update --init --recursive
No submodule mapping found in .gitmodules for path 'deps/bear'
The command "git submodule update --init --recursive" failed and exited with 1 during before_install.
Your build has been stopped.
[20:24:33] <Marc Worrell> oh, deps/bear!
[20:24:47] <Marc Worrell> bears? I hope swedish ones
[20:24:48] <Maas> is that on master?
[20:25:04] <Marc Worrell> yep
[20:25:04] <Marc Worrell> https://travis-ci.org/zotonic/zotonic/jobs/5507948
[20:25:16] <Maas> weird
[20:25:44] <Maas> oops
[20:25:59] <Maas> pressed the wrong merge button.
[20:26:12] <Arjan> omg
[20:26:21] <Maas> andreas had a pull for the stats branch
[20:26:27] <Arjan> I never use the github merge button
[20:26:44] <Arjan> find it too dangerous, plus you cannot do rebase before pulling
[20:26:49] <Maas> looking for the unpull button
[20:26:54] <Arjan> lol
[20:27:07] <Arjan> good luck ;)
[20:27:16] <Maas> omg
[20:27:35] <Maas> git unpull --omg
[20:28:14] <Arjan> I can try to fix it; what was the proper branch that neede pulling?
[20:28:28] <Maas> yes
[20:28:39] <Andreas Stenius> no branch was suposed to be pulled, at the moment (in zotonic/zotonic)
[20:28:48] <Andreas Stenius> there's two ways out
[20:29:02] <Maas> full of stuff that needed fixes for the new build
[20:29:03] <Andreas Stenius> either a forced push of master to reset it to the commit prior to the merge
[20:29:17] <Andreas Stenius> or to go through with the merge, and also merge the zotonic/webzmachine pull
[20:29:55] <Arjan> I would do the latter. just go for it.
[20:29:57] <Maas> eeks I don't even have that stuff cloned overhere.
[20:30:18] <Andreas Stenius> Arjan, I second that :)
[20:30:41] <Andreas Stenius> lets fix the bumps as they show up ;)
[20:30:43] <Maas> Just had the same problems as travis was having. rebar get-deps didn't work
[20:31:01] <Maas> got me an empty directory.
[20:31:11] <Andreas Stenius> hmm...
[20:31:20] <Arjan> forced pushes are evil
[20:31:21] Andreas Stenius heads over to the travis logs
[20:31:25] <Andreas Stenius> indeed
[20:31:37] <Maas> since we are on master now moving vm's
[20:31:44] <Andreas Stenius> being quick about it minimizes the risk that it spread, though
[20:31:52] <Maas> :-)
[20:32:21] <Maas> sorry guys
[20:33:04] <Arjan> np, just make sure you fix it :)
[20:33:34] <Andreas Stenius> I just merged the missing commits maas missed...
[20:33:47] <Maas> Thanks.
[20:34:34] <Andreas Stenius> and now also the webzmachine fixes from maas... those are needed for the stats to work
[20:34:45] <Arjan> ah, yes :)
[20:34:53] <Arjan> I was already about to give it a test drive
[20:35:00] <Andreas Stenius> :)
[20:35:01] <Maas> Did you cherry pick that one.
[20:35:16] <Andreas Stenius> uh? no, I took all your commits from your repo...
[20:35:38] <Andreas Stenius> https://github.com/zotonic/webzmachine/pull/10
[20:35:39] <Maas> the webzmachine repo
[20:35:44] <Andreas Stenius> yes
[20:36:43] <Maas> It just needed one cherry pick to work.
[20:36:50] <Maas> :p
[20:37:17] <Andreas Stenius> but the other changes... ? weren't they supposed to be committed?
[20:37:45] simon.smithies joins the room
[20:37:49] <Andreas Stenius> was it this commit that was the only one needed? https://github.com/zotonic/webzmachine/commit/051071460640f26621e0794ac2945c231985376f
[20:37:50] <Maas> Marc already did some of them on webzmachine.
[20:38:23] <Andreas Stenius> uhm, not quite (that I could see)... I had one conflict
[20:38:47] <Maas> on erlang:now I guess.
[20:39:13] <Andreas Stenius> ah, ok. well that got resolved so I didn't interfere...
[20:39:24] <Andreas Stenius> = I didn't notice.. :p
[20:39:40] <Maas> :-) We sort of made the same change
[20:40:02] arcusfelis leaves the room
[20:40:05] <Andreas Stenius> can you point to commits? (esp. yours)
[20:41:44] <Andreas Stenius> ah, are you talking about fada17bd006fcc7978cd0e413dd9e961a7b6b011 ? that one was merged already...
[20:41:58] <Maas> Looks they where exactly the same. I removed a couple of uses of erlang:now inside webzmachine. I don't see them in the pull.
[20:42:31] <Andreas Stenius> where did you commit them?
[20:42:54] <Maas> Looking.
[20:43:42] <Maas> https://github.com/mmzeeman/webzmachine/commit/fada17bd006fcc7978cd0e413dd9e961a7b6b011
[20:43:53] <Maas> Already merged I guess.
[20:44:37] <Andreas Stenius> yeah, that's the commit I posted at 20:37:17.. :)
[20:44:44] <Andreas Stenius> but in zotonic/webzmachine
[20:45:12] <Andreas Stenius> *link to commit...
[20:45:28] <Andreas Stenius> the travis build PASSED :D
[20:45:41] <Maas> but zotonic doesn't start here.
[20:45:47] <Andreas Stenius> what does it say?
[20:46:09] <Maas> do_boot case clause line 50 zotonic.erl
[20:46:18] <Andreas Stenius> I had to clean out a lot of old .beam files
[20:46:20] <Maas> {"init terminating in do_boot",{{case_clause,{error,{shutdown,{zotonic_app,start,[normal,[]]}}}},[{zotonic,ensure_started,1,[{file,"src/zotonic.erl"},{line,31}]},{zotonic,start,1,[{file,"src/zotonic.erl"},{line,50}]},{init,start_it,1,[]},{init,start_em,1,[]}]}}
[20:46:53] <Andreas Stenius> hmm... slightly different from what I saw when I was fiddling with it...
[20:48:02] <Maas> Nope, it doesn't start here.
[20:48:17] <Andreas Stenius> what does it say on the lines before that...
[20:48:38] <Andreas Stenius> it has to be some other previous error that leads to the application telling zotonic it should shutdown...
[20:49:02] <Andreas Stenius> perhaps we should handle that case so we can provide better error messages...
[20:49:23] <Andreas Stenius> do you have name clashes while booting?
[20:49:30] <Maas> not really I guess ensure started returns /= ok
[20:49:55] <Andreas Stenius> or does it say "** Found 0 name clashes in code paths" ?
[20:50:11] <Maas> No, oauth clashes
[20:50:15] <Andreas Stenius> no, it's application:start() that returns shutdown
[20:50:24] <Andreas Stenius> aha!
[20:50:41] <Andreas Stenius> rm modules/mod_oauth/deps/erlang-oauth/ebin/*
[20:51:02] <Maas> Does that prevent it from starting...
[20:51:16] vladimir.chujkov leaves the room
[20:51:30] <Andreas Stenius> I didn't look into it, but I had a bunch of name clashes, and that was one of them. It wouldn't work until I had resolved them...
[20:51:57] <Andreas Stenius> and the clashes are since rebar puts the .beam files in other places than our makefile used to...
[20:51:58] <Maas> new zotonic_deps module is doing that I see
[20:52:34] <Maas> Still not, no clashes now though
[20:52:49] <Maas> 20:52:12.644 [info] Application inets started on node zotonic001@buka
** Found 0 name clashes in code paths
20:52:12.733 [info] Application folsom started on node zotonic001@buka
20:52:12.798 [info] gen_smtp_server starting at zotonic001@buka
20:52:12.798 [info] gen_smtp_server listening on {0,0,0,0}:2525 via tcp
{"init terminating in do_boot",{{case_clause,{error,{shutdown,{zotonic_app,start,[normal,[]]}}}},[{zotonic,ensure_started,1,[{file,"src/zotonic.erl"},{line,31}]},{zotonic,start,1,[{file,"src/zotonic.erl"},{line,50}]},{init,start_it,1,[]},{init,start_em,1,[]}]}}
Crash dump was written to: erl_crash.dump
init terminating in do_boot ()
[20:53:24] <Andreas Stenius> hmm.. try a make all
[20:53:30] <Andreas Stenius> to get the latest dpes
[20:53:33] <Andreas Stenius> deps*
[20:53:50] <Andreas Stenius> esp the updated webzmachine will trigger a rebuild of almost all sources...
[20:53:55] <Maas> yup, it is doing that
[20:53:56] <Andreas Stenius> (or a lot of them, any way)
[20:55:19] <Maas> adding ?DEBUG
[20:55:32] <Maas> I don't see what is happening.
[20:56:09] <Andreas Stenius> no, it doesn't seem to be logging any thing relevant.
[20:56:41] <Andreas Stenius> but I'm wondering why there's a ensure_started/1 in zotonic_app.erl as well as in zotonic.erl..!
[20:57:46] <Andreas Stenius> if you don't have any local files or uncommitted changes you care about, a drastic way to get a clean checkout is to rm -rf * && git reset --hard
[20:58:00] <Andreas Stenius> that'll have the same effect as a fresh clone
[20:58:04] <Maas> No, clean
[20:58:28] <Andreas Stenius> got to go.. check in later (about 1 hr)
[20:58:36] <Maas> K.
[20:59:01] <Andreas Stenius> it does work for me... so it's most likely some mingling .beam file or out-of-sync dep or something...
[21:00:23] <Maas> K, i'll look in that direction first.
[21:00:32] simon.smithies leaves the room
[21:01:57] <Maas> First see what ensure_started returns.
[21:02:24] <Maas> :-) {error,{not_started,syntax_tools}}
[21:04:06] <Arjan> hmm
[21:04:11] <Arjan> I ought to have fixed that
[21:04:23] <Marc Worrell> yep - dat was fixed
[21:05:00] <Maas> Looking into it something is not working here.
[21:05:46] <Maas> Maybe I miss a package on ubuntu or something.
[21:06:00] <Maas> but syntax tools I have.
[21:06:36] <Arjan> are you on master?
[21:06:40] <Arjan> zotonic/zotonic master?
[21:06:45] <Maas> Yes.
[21:07:02] <Arjan> pulling
[21:07:18] <Maas> I get this: {"init terminating in do_boot",{{case_clause,{error,{shutdown,{zotonic_app,start,[normal,[]]}}}},[{zotonic,ensure_started,1,[{file,"src/zotonic.erl"},{line,33}]},{zotonic,ensure_started,1,[{file,"src/zotonic.erl"},{line,38}]},{zotonic,start,1,[{file,"src/zotonic.erl"},{line,52}]},{init,start_it,1,[]},{init,start_em,1,[]}]}}
[21:08:00] <Maas> The lines are off by 2 because I added ?DEBUG
[21:08:35] <Arjan> github is so slow the last week..
[21:10:36] <Maas> How much time does do_boot get?
[21:10:43] <Maas> 2 secs?
[21:10:55] simon.smithies joins the room
[21:11:12] <Marc Worrell> default of gen servers is 5 secs :p
[21:11:16] simon.smithies leaves the room
[21:11:23] simon.smithies joins the room
[21:11:55] <Maas> Hmm, I wonder if it just taking too long.
[21:12:30] <Arjan> my zotonic still starts fine after the last pull
[21:12:52] <Maas> :-) timing I think.
[21:13:21] <Maas> zotonic_deps does file io
[21:13:30] <Arjan> nice, those stats!!!!!!
[21:13:43] <Maas> Haha, I want stats too.
[21:14:25] <Arjan> but I don't get the graphs
[21:14:59] <Arjan> ah you see the average going down and up
[21:15:31] <Maas> You mean this bar charts. That is actually a histogram
[21:15:37] <Maas> of the durations.
[21:16:57] <Arjan> I dont know
[21:17:35] <Arjan> I should show you guys the stats views we have in zynamo. those make more sense to me.
[21:18:30] <Arjan> its probably just a matter of what gets displayed
[21:19:13] <Maas> This is more or less represents straight what is collected. Probably needs some some work to make more sense indeed.
[21:20:47] <Arjan> should probably collect requests / sec
[21:20:52] <Arjan> anyhow
[21:20:55] <Arjan> i'm off for the night
[21:21:59] <Maas> It does that I think. Haven't seen the graphs myself yet.
[21:22:10] <Arjan> heh
[21:22:11] <Arjan> ok
[21:22:24] <Arjan> when you see them, you probably understand what I mean.
[21:22:27] <Arjan> g'night
[21:22:31] Arjan leaves the room
[21:22:37] <Maas> Good night
[21:29:55] Arjan joins the room
[21:38:51] vladimir.chujkov joins the room
[21:44:12] <Maas> It looks like do_boot times out on my setup. zotonic_sup:init finishes, but I never get a pid back in zotonic_sup:start_link...
[21:47:33] vladimir.chujkov leaves the room
[21:48:08] <Marc Worrell> we need a better boot sequence
[21:49:34] <Maas> It looks like it is capped somewhere at 2 seconds.
[21:49:47] <Marc Worrell> strange
[21:49:53] <Marc Worrell> that is way too short
[21:49:59] <Marc Worrell> we need startup phases
[21:50:08] <Marc Worrell> first start the system
[21:50:12] <Marc Worrell> then one by one the sites
[21:50:41] <Maas> Something like that. Mnesia does something like that too.
[21:51:59] <Maas> The vm setup I use has very slow disk speed on the shared (between host and vm) folder zotonic is sitting in.
[21:52:32] <Marc Worrell> good test case
[21:53:07] <Marc Worrell> I think Arjan will find some nice solutions when he starts using it on Raspberry Pi
[21:53:33] <Maas> So if there is some file scanning somewhere then it will probably hit a limit
[21:53:57] <Maas> I changed the module indexer so it indexes in a separate process
[21:54:11] <Maas> so lookups can continue while it is scanning
[21:54:41] <Maas> Saw that happening on the atom too.
[21:55:19] <Maas> With the vm setup things are kind of different. The cpu is fast, but disk access is very slow.
[21:58:11] <Marc Worrell> github is slooooow today
[22:03:05] <simon.smithies> My 0.9 zotonic is kind of randomly dying ... I get logged out of the sites admin page and when I log back in all my sites have stopped
[22:03:19] <simon.smithies> any tips on finding the cause?
[22:03:47] <simon.smithies> priv/sasl and priv/log have nothing about a crash or anything in them
[22:04:21] <Marc Worrell> nothing at all?
[22:04:45] <Marc Worrell> and the console logs?
[22:05:08] <simon.smithies> well .. I have tail on crash.log .. seeing nothing there
[22:05:21] <simon.smithies> watching error.log now
[22:05:45] <simon.smithies> is there another place to look
[22:05:47] <simon.smithies> ?
[22:06:02] <Marc Worrell> sasl is the place to be
[22:06:28] <Marc Worrell> what are your settings in the vm config?
[22:06:59] <Marc Worrell> priv/erlang.config
[22:07:22] <simon.smithies> I have no idea! where do I find my vm config?
[22:07:28] <simon.smithies> sorry i see
[22:07:31] <Marc Worrell> :-)
[22:07:38] <Marc Worrell> I had to look it up as well
[22:08:53] <Marc Worrell> the erlang.config.in gives a good start for that file
[22:09:16] <Marc Worrell> My development env has this one:
[22:09:17] <Marc Worrell> [
{lager, [
%% What handlers to install with what arguments
{handlers, [
{lager_console_backend, [info, true]},
{lager_file_backend, [
{"priv/sasl/error.log", error, 10485760, "$D0", 5},
{"priv/sasl/console.log", info, 10485760, "$D0", 5}
%% Whether to write a crash log, and where. Undefined means no crash logger.
{crash_log, "priv/sasl/crash.log"},
%% Maximum size in bytes of events in the crash log - defaults to 65536
{crash_log_msg_size, 65536},
%% Maximum size of the crash log in bytes, before its rotated, set
%% to 0 to disable rotation - default is 0
{crash_log_size, 10485760},
%% What time to rotate the crash log - default is no time
%% rotation. See the README for a description of this format.
{crash_log_date, "$D0"},
%% Number of rotated crash logs to keep, 0 means keep only the
%% current one - default is 0
{crash_log_count, 5},
%% Whether to redirect error_logger messages into lager - defaults to true
{error_logger_redirect, true}
[22:10:31] <simon.smithies> ok .. mine has %% -*- mode: erlang -*-
[{lager_console_backend, info},
[{"priv/sasl/error.log", error, 10485760, "$D0", 5},
{"priv/sasl/console.log", info, 10485760, "$D0", 5}
{crash_log, "priv/sasl/crash.log"}
[{wmtrace_dir, "priv/wmtrace"},
{silent_console, true}
[22:11:42] <simon.smithies> so mine looks pretty good?
[22:14:58] <simon.smithies> ah! got an error
[22:15:02] <simon.smithies> [error] <0.199.0>@z_webmachine_error_handler:render_error:33 webmachine error: path="/postback": {error,{error,{badmatch,false},[{controller_zotonic_status,event,2,[{file,"priv/sites/zotonic_status/controllers/controller_zotonic_status.erl"},{line,112}]},{controller_postback,process_postback,1,[{file,"modules/mod_base/controllers/controller_postback.erl"},{line,144}]},{controller_postback,process_post,2,[{file,"modules/mod_base/controllers/controller_postback.erl"},{line,70}]},{webmachine_controller,controller_call,3,[{file,"src/webmachine_controller.erl"},{line,200}]},{webmachine_decision_core,decision,3,[{file,"src/webmachine_decision_core.erl"},{line,484}]},{webmachine_decision_core,handle_request,2,[{file,"src/webmachine_decision_core.erl"},{line,31}]},{webmachine_mochiweb,loop,2,[{file,"src/webmachine_mochiweb.erl"},{line,106}]},{mochiweb_http,headers,5,[{file,"src/mochiweb_http.erl"},{line,136}]}]}}
[22:15:57] <Arjan> apparently you are not logged in
[22:16:00] <simon.smithies> in error.log
[22:16:14] <Arjan> that line reads:
true = z_auth:is_auth(Context),
[22:16:29] <simon.smithies> I'm not now
[22:17:57] <Marc Worrell> that error typically happens when you have a tab open, after the system restarted
[22:18:04] <Marc Worrell> at least it is logging something now!
[22:18:33] <simon.smithies> it's just done it again
[22:19:07] <simon.smithies> but nothing new in error.log
[22:19:29] <Marc Worrell> and in console.log?
[22:20:06] <simon.smithies> same old start up stuff
[22:20:27] <simon.smithies> 2013-03-15 10:17:50.600 [info] <0.149.0> gen_smtp_server listening on {0,0,0,0}:2525 via tcp
2013-03-15 10:17:50.687 [info] <0.6.0> Application zotonic started on node zotonic001@vps783
2013-03-15 10:17:54.572 [info] <0.142.0>@zotonic_sup:init:171
2013-03-15 10:17:54.572 [info] <0.142.0>@zotonic_sup:init:172 Zotonic started
2013-03-15 10:17:54.572 [info] <0.142.0>@zotonic_sup:init:173 ===============
2013-03-15 10:17:54.572 [info] <0.142.0>@zotonic_sup:init:174 Web server listening on IPv4 any:8000
2013-03-15 10:17:54.572 [info] <0.142.0>@zotonic_sup:init:176 Web server listening on IPv6 ::8000
2013-03-15 10:17:54.573 [info] <0.142.0>@zotonic_sup:init:180
[22:20:31] <Marc Worrell> then your session got killed - for whatever reason
[22:20:36] <simon.smithies> yep
[22:21:07] <Marc Worrell> the session timeout is quite short - but you can play with it
[22:21:09] <simon.smithies> but latest in error.log is still that auth message
[22:21:34] <simon.smithies> which is actually from before the startup
[22:21:48] <simon.smithies> so that was from last time -- red herring
[22:22:11] <Marc Worrell> indeed - i was just now chasing an error from last februari...
[22:22:20] <simon.smithies> :)
[22:22:24] <Marc Worrell> log was switched off
[22:23:19] <simon.smithies> if admin page session times out, that shouldn't kill my running sites, should it?
[22:23:28] <Marc Worrell> no
[22:23:45] <Marc Worrell> but when a site crashes then it should leave a message in the error log
[22:24:58] <simon.smithies> I wish I could repeat whatever it is that causes everything to stop
[22:25:10] <simon.smithies> but it seems to be random
[22:25:34] <Marc Worrell> very strange indeed - I have never seen it like that - without any errors
[22:26:04] <Marc Worrell> saw one time a total crash after the ssl app gave up - it dragged quite a bit with it - but also a lot of noise in the logs
[22:26:28] <simon.smithies> the admin site has a Toggle Tracing button ... does that put more information somewhere?
[22:26:52] <simon.smithies> I don't think I have SSL enabled
[22:27:07] <simon.smithies> that's a module I can turn on per site?
[22:28:40] <simon.smithies> looking for the SSL module in one of my sites, I notice mod_backup is activated
[22:29:04] <simon.smithies> thought I'd switched it off because of the REST dependency ... going to work through that later
[22:29:30] <simon.smithies> but anyway ... clicked deactivate and got FAIL: error
[22:32:37] <simon.smithies> site went down again ... has come back up and now I can deactivate that module
[22:32:41] <simon.smithies> aaaargh
[22:32:54] <Marc Worrell> could you add some debug to z_supervisor:notify_exit ?
[22:33:22] <Marc Worrell> and set the debug level in your console to 'debug'
[22:33:23] <simon.smithies> will need you to talk me through it, but yep
[22:33:56] <Marc Worrell> in src/support/z_supervisor.erl
[22:34:03] <Marc Worrell> around line 550
[22:34:41] <Marc Worrell> %% @doc Notify the manager that a child has stopped
notify_exit(_ChildState, #state{manager_pid=undefined}) ->
notify_exit(ChildState, #state{manager_pid=ManagerPid}) ->
gen_server:cast(ManagerPid, {supervisor_child_stopped, ChildState#child_state.child, ChildState#child_state.pid}).
[22:34:53] <Marc Worrell> there is no manager defined for the sites supervisor
[22:35:11] <Marc Worrell> so just before the nop you could add some debug
[22:35:43] <simon.smithies> ok
[22:36:00] <Marc Worrell> ?DEBUG(_ChildState)
[22:36:06] <Marc Worrell> and a comma
[22:36:13] <Marc Worrell> ?DEBUG(_ChildState),
[22:36:29] <Marc Worrell> then set the log level of your console.log to debug
[22:36:34] <Marc Worrell> make
[22:36:37] <Marc Worrell> and restart
[22:38:16] <simon.smithies> like this {"priv/sasl/console.log", debug, 10485760, "$D0", 5}
[22:38:19] <simon.smithies> ?
[22:38:26] <Marc Worrell> yes
[22:38:45] <Marc Worrell> is this a production or a test server?
[22:38:51] <simon.smithies> test :)
[22:38:55] <Marc Worrell> pfew :p
[22:39:12] <Marc Worrell> we are running 0.9.x in production - so it is quite weird
[22:39:47] <simon.smithies> brave
[22:40:01] <simon.smithies> I guess you're not haveing issues like this though
[22:40:17] <Marc Worrell> no, we haven't.
[22:40:22] <Marc Worrell> interesting
[22:40:36] <Marc Worrell> actually we are running the 0.9 since june in production
[22:40:57] <Marc Worrell> sustained quite a bit of traffic
[22:41:08] <simon.smithies> that is good news
[22:41:48] <Marc Worrell> 60K visitors in less than a day on one site and mailinglists of 20K+
[22:42:30] <Marc Worrell> and all together in the same beam process as zotonic.com
[22:42:57] <simon.smithies> based on what I've seen in 0.7 and 0.8, it's going to be great on 0.9 ... just have to get these things sorted
[22:43:09] <simon.smithies> really?
[22:43:38] <simon.smithies> that really supports my thought that I can run lots of little client sites on the same instance
[22:44:29] <simon.smithies> console.log is really interesting now :)
[22:44:43] <simon.smithies> zotonic died, then came back up again
[22:44:57] <Arjan> can you put it in a pastebin?
[22:45:48] <simon.smithies> pastebin? I need to get into the new millenium ;)
[22:46:12] <Arjan> http://pastebin.com/
[22:47:46] <Marc Worrell> adding some extra logging to z_sites_manager
[22:47:47] <simon.smithies> http://pastebin.com/xQ1wFeZm
[22:48:01] <simon.smithies> that is from first startup
[22:48:11] <simon.smithies> and I notice it's going down again as I write this
[22:48:49] <Marc Worrell> better delete that pastebin soon - I see a password :p
[22:49:08] <simon.smithies> well spotted
[22:49:15] <simon.smithies> you got it?
[22:49:35] <Marc Worrell> is a normal startup sequence - as far as I can see
[22:49:52] <Marc Worrell> any new entries from around your restart?
[22:50:09] <Arjan> I dont see any error either
[22:50:20] <Arjan> pastebin is reeeaaly slow for me
[22:50:32] <simon.smithies> no, the new start looks just ike the last 2
[22:50:46] <simon.smithies> I just deleted that one
[22:50:53] <simon.smithies> yo uwant it back?
[22:51:44] <Marc Worrell> we are interested in any messages after Zotonic says it is running
[22:51:57] <Marc Worrell> (the part with the sitenames and port numbers)
[22:56:03] <simon.smithies> this is what I see when I start a site
[22:56:04] <simon.smithies> http://pastebin.com/Qwc2FhRB
[22:58:22] Arjan leaves the room
[23:02:19] <simon.smithies> so it crashed again ... oddly the log entries following the crash are timestamped 2 minutes earlier
[23:02:38] <Marc Worrell> weird
[23:03:05] <Marc Worrell> I just added a site start/stop into message to z_sites_manager
[23:03:14] <Marc Worrell> into = info
[23:03:37] <simon.smithies> this is following the crash http://pastebin.com/0aj5MrfB
[23:04:20] <Marc Worrell> wow - that is a complete restart of the vm!
[23:05:34] Andreas Stenius leaves the room
[23:05:40] <Marc Worrell> are you running zotonic in debug?
[23:05:46] <Marc Worrell> or with heart?
[23:06:24] <simon.smithies> with heart
[23:07:16] <Marc Worrell> hmmm, maybe something external is crashing it
[23:07:22] <Marc Worrell> can you run it in debug?
[23:07:33] <simon.smithies> yep
[23:07:44] <Maas> oom in linux?
[23:07:55] <simon.smithies> ?
[23:08:15] <Marc Worrell> can you also check if there is a erl_crash.dump ?
[23:08:19] <Maas> That is an out of memory deamon which kills off processes
[23:08:22] <Marc Worrell> oom = out of memory
[23:08:31] <simon.smithies> ah
[23:08:32] <Marc Worrell> how big is your vps?
[23:08:45] <Maas> I've seen it kill off postgres too.
[23:08:50] <Marc Worrell> yes?
[23:08:51] <simon.smithies> let me check
[23:09:04] <Marc Worrell> wow, killing postgres
[23:09:12] <simon.smithies> is bigger than than the one where I run 0.8 with the same sites on it
[23:09:50] <Maas> That is not fun.. indeed. We had a lot of visitors at the time (and a bug)
[23:10:05] <simon.smithies> 768MB memory
[23:11:41] <simon.smithies> where would erl_crash.dmp be
[23:12:09] <simon.smithies> found it
[23:12:16] <Marc Worrell> 768 is plenty
[23:12:26] <Marc Worrell> (or should be)
[23:12:52] <Marc Worrell> I have run it with less, but it was a 32 bit linux
[23:12:54] <simon.smithies> shall I pastebin erl_crash.dump?
[23:13:08] <Marc Worrell> when you use 64 bits then it memory goes fast
[23:13:12] <Marc Worrell> yes, pastebin it
[23:13:23] <simon.smithies> I'm serving 6 little sites happily of 32bit debian on 512MB
[23:14:58] <Marc Worrell> .9 uses a bit more memory (more apps) but is not that much different that .8
[23:16:30] <simon.smithies> showing my shell ignorance here ... how do I get the content of erlang_crash.dump from a terminal into my os x clipboard
[23:16:51] <Marc Worrell> lol, good question
[23:17:00] <Marc Worrell> i would open it in my text editor
[23:21:30] <Marc Worrell> I am calling it a night - at 6am my son is getting up again :p
[23:21:44] <Marc Worrell> paste the log - and I will have a look tomorrow
[23:21:49] <simon.smithies> OK -- thanks for the help!!
[23:21:54] <simon.smithies> will do
[23:22:13] <simon.smithies> sleep well
[23:29:41] <simon.smithies> erl_crash.dump: http://pastebin.com/rkpu6NaZ
Powered by ejabberd Powered by Erlang Valid XHTML 1.0 Transitional Valid CSS!