Monday, 14 January 2013< ^ >
arjan has set the subject to: Zotonic - the Erlang Content Management Framework
Room Configuration

[07:33:15] Arjan joins the room
[08:26:58] maas.maarten.zeeman joins the room
[08:46:21] Arjan leaves the room
[08:52:58] andreas.stenius joins the room
[08:57:15] Arjan joins the room
[09:11:48] <maas.maarten.zeeman> Sigh... installing new site fails. Can connect the db, and have a fresh db, but then it dies with 'relation "config" does not exist.' error. Weird.
[09:14:50] <Arjan> oh?
[09:15:01] <Arjan> on your db branch, I hope :P
[09:16:04] <maas.maarten.zeeman> No zotonic master.
[09:17:33] <maas.maarten.zeeman> dropping db, and doing it again. Maybe i made a typo somewhere.
[09:17:37] <Arjan> wow, strange
[09:17:45] <Arjan> does the db schema exist?
[09:18:29] <maas.maarten.zeeman> no tables where created
[09:19:39] <maas.maarten.zeeman> how can you check if the schema is there?
[09:20:31] <andreas.stenius> psql \dn I think it was..
[09:21:16] <maas.maarten.zeeman> it says that public schema exists, so that is fine.
[09:22:34] <andreas.stenius> no hints in the logs?
[09:23:25] <maas.maarten.zeeman> Strange, dropping db, recreating it and recompiling doesn't solve problem....
[09:24:12] <maas.maarten.zeeman> It just get this: SQL error {error,{error,error,<<"42P01">>,<<"relation "config" does not exist">>,[{position,<<"15">>}]}} : "select * from config order by module, key"
[09:24:30] <Arjan> maybe some process is trying to get something from the config too soon?
[09:24:36] <maas.maarten.zeeman> prob some typo.
[09:24:38] <Arjan> do you have a stack trace?
[09:25:03] <Arjan> maybe it is your user agent commit?
[09:25:17] <andreas.stenius> was thinking that too..
[09:25:52] <maas.maarten.zeeman> how?
[09:26:53] <Arjan> how what?
[09:27:08] <maas.maarten.zeeman> how could the ua_classifier affect the db?
[09:27:36] <Arjan> no idea
[09:28:48] <Marc Worrell> Last week I installed a new site, in an empty schema. In fact repeated it many times for testing the mod_survey activate problem.
[09:28:53] <Marc Worrell> Worked like a charm
[09:28:54] <andreas.stenius> I've not looked at it, but you added a config option to the init procedure of ua_classifier, did you not? That config lookup hopefully doesn't do a db roundtrip, does it?
[09:29:24] <Marc Worrell> Uh oh - that is interesting - I did test it before that change
[09:29:37] <maas.maarten.zeeman> hmm. lets see.
[09:29:38] <Marc Worrell> does it use z_config or m_config?
[09:29:43] <maas.maarten.zeeman> z_config
[09:29:52] <maas.maarten.zeeman> so it should be the file
[09:29:56] <andreas.stenius> ok
[09:29:59] <Marc Worrell> z_config shouldn't check the db
[09:30:09] <andreas.stenius> not that then :)
[09:30:24] <maas.maarten.zeeman> it fails during installation. Here is what it does:
[09:30:37] <andreas.stenius> patch the SQL routine to give a stack traceback in case of errors.. ! ;)
[09:30:59] <maas.maarten.zeeman> 09:30:36.762 [warning] webshotmewww: Installing database webshot.public at webshot@
09:30:37.721 [info] SQL error {error,{error,error,<<"42P01">>,<<"relation "config" does not exist">>,[{position,<<"15">>}]}} : "select * from config order by module, key"
09:30:38.694 [info] SQL error {error,{error,error,<<"42P01">>,<<"relation "config" does not exist">>,[{position,<<"15">>}]}} : "select * from config order by module, key"
09:30:39.696 [info] SQL error {error,{error,error,<<"42P01">>,<<"relation "config" does not exist">>,[{position,<<"15">>}]}} : "select * from config order by module, key"
09:30:40.588 [info]
[09:31:29] <andreas.stenius> hmm... or turn on query logging to see every query issued during initialization...
[09:31:37] <maas.maarten.zeeman> postgres=# \c webshot;
psql (8.4.15)
You are now connected to database "webshot".
webshot=# create schema public;
ERROR: schema "public" already exists
[09:31:54] <andreas.stenius> http://zotonic.com/docs/0.9/manuals/cookbook/shell-debuggingdatabase.html
[09:33:08] <Marc Worrell> schema public is always created automatically
[09:33:43] <Marc Worrell> check who owns the schema and if there are any tables, who owns those tables
[09:33:57] <maas.maarten.zeeman> hmm. most logical is a simple typo somewhere.
[09:34:00] <Arjan> yeah might be a permission problem
[09:34:07] <Marc Worrell> sometimes a user can't see what is somewhere because of permissions
[09:34:08] <Arjan> if you do \dn in PG
[09:34:15] <Arjan> you see who owns the public schema
[09:34:20] <Marc Worrell> and then the error is "not found" instead of "no access"
[09:37:26] <maas.maarten.zeeman> owner is postgres
[09:39:28] <maas.maarten.zeeman> Same for the zotonic db.
[09:39:30] <maas.maarten.zeeman> weird
[09:39:45] <Marc Worrell> en je logt in als?
[09:40:03] <maas.maarten.zeeman> sudo -u postgres psql
[09:40:16] <andreas.stenius> and from zotonic as?
[09:40:21] <maas.maarten.zeeman> also
[09:40:44] <andreas.stenius> as postgres from zotonic too?
[09:40:49] <maas.maarten.zeeman> yes.
[09:40:53] <andreas.stenius> ah, ok
[09:41:00] <maas.maarten.zeeman> lets see if that site still works.
[09:41:00] <Marc Worrell> strange
[09:41:08] <Marc Worrell> and are the tables there?
[09:41:32] <andreas.stenius> that was the thing, no tables were created..
[09:41:44] <Marc Worrell> maybe check the information schema? see what that one says - as we query that schema to know if we have to create something or not
[09:42:01] <maas.maarten.zeeman> No tables indeed. The infromation scheme just lists the 133! default tables
[09:42:25] <Marc Worrell> so simple and small, that progresql :p
[09:43:13] <Marc Worrell> maybe you can check the spot where we check the existence of config on startup?
[09:43:43] <maas.maarten.zeeman> yup, checking now if using an existing db still works.
[09:44:17] <maas.maarten.zeeman> but I need some patience.... recompiling stuff on atom netbook takes ages...
[09:44:24] <maas.maarten.zeeman> cd
[09:44:36] <Marc Worrell> LOL
[09:45:04] <Marc Worrell> is a good check - we need more testing on slow stuff - and some rethinking of some timeouts and structure
[09:45:27] <andreas.stenius> indeed.
[09:45:30] <maas.maarten.zeeman> I love this netbook thingy. :-)
[09:45:33] <Marc Worrell> would be great if we can become a good cms for simple (distributed) hardware
[09:46:07] <andreas.stenius> yeah. I'd love to run zotonic on my DD-WRT router :D
[09:46:21] <Marc Worrell> Was thinking what would happen if you just connect a big bunch of Raspberry Pi's together in a rack mounted pizza box
[09:46:40] <andreas.stenius> lol- pizza box :)
[09:46:41] <maas.maarten.zeeman> haha. me too.
[09:46:48] <Marc Worrell> Zotonic on your router - now that is efficient use of infrastructure!
[09:47:03] <andreas.stenius> exactly! :)
[09:47:21] <maas.maarten.zeeman> maybe wait until the pi's have 1024 mb ram.
[09:47:23] <Marc Worrell> can someone make a ethernet powered raspberry pi? (or clone)
[09:47:52] <Marc Worrell> With PoE you have a poor man's blade server :p
[09:47:59] <andreas.stenius> I've not looked into this raspberry pi thing yet...
[09:48:01] <maas.maarten.zeeman> Or the mobo's of those samsung chromebooks
[09:48:22] <Marc Worrell> there are some raspberry pi alternatives - with more GBs and GHzs
[09:48:57] <Marc Worrell> but you also need more checksums on your data - no ECC RAM....
[09:50:37] <maas.maarten.zeeman> existing site works just fine.
[09:51:25] <Marc Worrell> so it might be the "does config exist" query on the information schema - that is always returns "sure ok, go ahead, make my day"
[09:51:36] <Marc Worrell> is = it
[09:54:31] <andreas.stenius> "Is power over Ethernet (PoE) possible?
Not in the base device, but it’s been a very commonly requested feature, so we’re examining options for later releases." -http://www.raspberrypi.org/faqs
[09:55:59] <maas.maarten.zeeman> :q
[09:56:06] <maas.maarten.zeeman> oops
[09:59:30] <maas.maarten.zeeman> Adding debug to has_table...
[10:02:18] <maas.maarten.zeeman> 10:01:02.191 [info] DEBUG: z_installer:87 "\n select count(*) \n from information_schema.tables \n where table_catalog = $1 \n and table_name = $3 \n and table_schema = $2\n and table_type = 'BASE TABLE'"
10:01:02.212 [info] DEBUG: z_installer:87 {ok,0}
[10:02:22] <maas.maarten.zeeman> So no db.
[10:02:29] <maas.maarten.zeeman> no config table.
[10:03:27] <Marc Worrell> then it should proceed installing it. Is it?
[10:04:05] <maas.maarten.zeeman> it is trying too.
[10:11:55] <Marc Worrell> maybe a timeout then?
[10:12:09] <Marc Worrell> we really need start phases....
[10:12:41] <maas.maarten.zeeman> weird..
[10:12:43] <andreas.stenius> the OTP app has support for start phases...
[10:12:53] <maas.maarten.zeeman> it runs through all the create steps...
[10:13:06] <maas.maarten.zeeman> 10:08:43.691 [info] DEBUG: z_install:85 "ALTER TABLE rsc ADD CONSTRAINT fk_rsc_creator_id FOREIGN KEY (creator_id)\n REFERENCES rsc (id)\n ON UPDATE CASCADE ON DELETE SET NULL"
10:08:43.700 [info] DEBUG: z_install:85 {ok,[],[]}
10:08:43.701 [info] DEBUG: z_install:85 "ALTER TABLE rsc ADD CONSTRAINT fk_rsc_modifier_id FOREIGN KEY (modifier_id)\n REFERENCES rsc (id)\n ON UPDATE CASCADE ON DELETE SET NULL"
10:08:43.711 [info] DEBUG: z_install:85 {ok,[],[]}
10:08:43.712 [info] DEBUG: z_install:85 "CREATE INDEX fki_rsc_creator_id ON rsc (creator_id)"
10:08:43.798 [info] DEBUG: z_install:85 {ok,[],[]}
10:08:43.799 [info] DEBUG: z_install:85 "CREATE INDEX fki_rsc_modifier_id ON rsc (modifier_id)"
[10:13:23] <Marc Worrell> yes, but I think we might come up with some flexible stuff for those phases than an OTP app - we also need it for starting modules etc (multiple times per zotonic app)
[10:13:28] <maas.maarten.zeeman> the last of the bunch... only no config table afterwards.
[10:13:40] <Marc Worrell> maybe a rollback?
[10:13:59] <Marc Worrell> everything is done inside a transaction
[10:14:04] <maas.maarten.zeeman> checking commit...
[10:14:15] <Marc Worrell> could be that you are getting a late timeout on the commit
[10:14:36] <maas.maarten.zeeman> there has been a ubuntu postgres update last week.
[10:14:37] <Arjan> hmm
[10:14:50] <Arjan> somehow its always these people with slow laptops that have problems installing zotonic :p
[10:15:35] <maas.maarten.zeeman> haha\
[10:15:39] <andreas.stenius> I thought you were going to say that it always these people updating stuff from some central package repo that breaks our stuff :p
[10:16:15] <Marc Worrell> LOL
[10:16:45] <Marc Worrell> I should bring my small ARM machine back to life, then I can also check this kind of things.
[10:18:07] <maas.maarten.zeeman> the trimslice.
[10:18:52] <maas.maarten.zeeman> something is masking the real error here.
[10:19:34] <Marc Worrell> hmmmm, strange. maybe check on a different db connection, after the commit, if the table is there?
[10:21:21] <maas.maarten.zeeman> Now checking z_install with_transaction function. That always returns ok without checking responses from z_install_data
[10:25:03] <Arjan> ah
[10:25:21] <maas.maarten.zeeman> so.... checking what i get.
[10:25:30] <Marc Worrell> I guess it assumes that z_install_data should crash when not installing?
[10:25:40] <Marc Worrell> the installer checks everywhere on 'ok'
[10:28:32] <maas.maarten.zeeman> It is getting even stranger now... install data reports install done (it didn't do that before)
[10:29:06] <andreas.stenius> postgres' scrubbed the discs warm enough to go through?
[10:29:30] <Marc Worrell> must be something simple we are overlooking
[10:29:41] <andreas.stenius> it usually is :p
[10:30:04] <maas.maarten.zeeman> it started.
[10:30:16] <maas.maarten.zeeman> weird
[10:30:20] <Marc Worrell> strange….
[10:31:12] <Marc Worrell> packing, have to go to the Maximonster head quarters in town
[10:31:29] <maas.maarten.zeeman> :-)
[10:31:41] <maas.maarten.zeeman> i don't get it.
[10:32:43] <maas.maarten.zeeman> maybe it stops because starting the zotonic application takes too long
[10:32:57] Marc Worrell leaves the room: Disconnected: session closed
[10:33:12] <andreas.stenius> could very well be the case.
[10:33:21] <maas.maarten.zeeman> I got this in the last try:
[10:33:24] <maas.maarten.zeeman> 10:27:21.815 [info] DEBUG: z_install_data:248 "Inserting predicates"
10:27:21.913 [info] DEBUG: z_install_data:42 {webshotmewww,"Install done."}
10:27:21.914 [info] DEBUG: z_install:63 ok
10:27:21.934 [info] DEBUG: z_install:61 ok
Terminating due to shutdown
{"init terminating in do_boot",{{badmatch,{error,{shutdown,{zotonic_app,start,[normal,[]]}}}},[{zotonic,start,1},{init,start_it,1},{init,start_em,1}]}}
Crash dump was written to: erl_crash.dump
init terminating in do_boot ()
[10:33:26] <andreas.stenius> in that case, the one timing out really ought to say something about it...
[10:33:41] <maas.maarten.zeeman> why did it terminate?
[10:33:51] <andreas.stenius> inspect the crash dump... ?
[10:34:48] <andreas.stenius> the timeout hit it, perhaps.. ?
[10:35:37] <maas.maarten.zeeman> It just says terminating on do_boot().
[10:36:05] <andreas.stenius> hmm... maybe a badmatch error in do_boot always causes a dump...
[10:36:39] <maas.maarten.zeeman> =erl_crash_dump:0.1
Mon Jan 14 10:27:22 2013
Slogan: init terminating in do_boot ()
System version: Erlang R14B04 (erts-5.8.5) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:true]
Compiled: Thu Nov 3 08:07:35 2011
Taints: crypto,ua_classifier
Atoms: 14175
total: 55133344
processes: 43391168
processes_used: 43376936
[10:36:47] <andreas.stenius> This is a so common dump, we really ought to match on the error shutdown and shutdown cleaner..
[10:37:05] <maas.maarten.zeeman> wow... why that many processes...
[10:37:22] <andreas.stenius> no, I think that is memory used by processes...
[10:37:41] <maas.maarten.zeeman> o
[10:37:52] <andreas.stenius> but still, seems like quite a bit
[10:41:14] <maas.maarten.zeeman> documentations says that if you get this error the primitive boot sequence was terminated.
[10:45:52] <maas.maarten.zeeman> That narrows things down quite a bit.
[10:48:15] <maas.maarten.zeeman> zotonic:start....
[10:53:44] <maas.maarten.zeeman> So it times out in the boot sequence.
[10:54:10] <andreas.stenius> as Marc said, we need to use start phases..
[10:54:32] <andreas.stenius> either those from OTP app, or some homegrown
[10:57:17] <maas.maarten.zeeman> Sounds logical.
[11:03:20] Arjan leaves the room
[11:03:40] <maas.maarten.zeeman> btw mnesia (which can also take a long time before it start) just waits until the supervisor is started...
[11:07:57] <maas.maarten.zeeman> ow and that starts the whole shebang.
[11:08:52] <andreas.stenius> yep, really time to go over the startup functionality...
[11:09:15] <maas.maarten.zeeman> damn yes.
[11:09:18] <maas.maarten.zeeman> :-)
[11:09:43] <maas.maarten.zeeman> or extend a timeout from 5 secs to something bigger.
[11:10:09] <andreas.stenius> I think it's ok to have a quick startup (the 5 sec), just that things that take time shouldn't go in there...
[11:12:18] <maas.maarten.zeeman> Looking at mnesia right now.
[11:12:36] <andreas.stenius> are you using it?
[11:12:52] <andreas.stenius> I get mixed signals regarding mnesia..
[11:13:03] <andreas.stenius> on one hand, it's said to be great..
[11:13:09] <maas.maarten.zeeman> I have been in the past. That can take some time to get started.
[11:13:16] <andreas.stenius> on the other hand, it's unusable (for large amounts of data, at least)
[11:13:17] <maas.maarten.zeeman> Was checking how it does that
[11:13:51] <maas.maarten.zeeman> Zotonic uses it too ;-)
[11:13:54] <andreas.stenius> sounds good. Those who wrote it ought to now how to do it properly.. :p
[11:14:13] <andreas.stenius> I though zotonic only used ets.. ?
[11:14:18] <andreas.stenius> thought*
[11:14:29] <maas.maarten.zeeman> mnesia for email sending
[11:14:35] <andreas.stenius> oh..
[11:14:50] <andreas.stenius> what do you.. :p
[11:14:57] <andreas.stenius> ... know
[11:15:34] <maas.maarten.zeeman> Sigh... have to get started with actual work :-) Otherwise I get a timeout too.
[11:16:02] <andreas.stenius> lol- true.
[11:16:48] <maas.maarten.zeeman> Used mnesia in distributed setup. Pretty cool and nice. But when things go wrong they go wrong very badly.
[11:17:15] <maas.maarten.zeeman> Like network splits...
[11:17:19] <maas.maarten.zeeman> eeks
[11:17:54] <andreas.stenius> ouch
[11:17:55] <maas.maarten.zeeman> It is fine to use it on a single machine. Then having atomic updates is very handy. This is missing from ets.
[11:30:17] Arjan joins the room
[11:57:01] Marc Worrell joins the room
[12:46:24] Arjan leaves the room
[12:46:26] Marc Worrell leaves the room: Disconnected: session closed
[13:23:01] Arjan joins the room
[13:24:15] Marc Worrell joins the room
[13:33:19] <maas.maarten.zeeman> btw. why does zotonic do a build when starting?
[13:34:47] <maas.maarten.zeeman> for submodules it seems to store the name of the root directory somehow.
[13:35:01] <maas.maarten.zeeman> when i rename the directory it doesn't work anymore.
[13:37:11] <maas.maarten.zeeman> so if i first compile in zotonic-zotonic
[13:37:13] <Arjan> beh
[13:37:23] <Arjan> git submodules ....
[13:37:30] <maas.maarten.zeeman> and then rename it to zotonic it looks up submodules in zotonic-zotonic
[13:37:39] <maas.maarten.zeeman> aha gotcha
[13:39:18] <maas.maarten.zeeman> ln -s to the rescue
[14:04:18] <Arjan> :-/
[14:05:57] <maas.maarten.zeeman> for quickly switching between branches.
[17:33:50] Marc Worrell leaves the room: Disconnected: session closed
[17:34:31] Arjan leaves the room
[17:37:20] maas.maarten.zeeman leaves the room
[18:01:58] Protagores joins the room
[19:24:38] Arjan joins the room
[20:21:26] Marc Worrell joins the room
[20:32:30] <Arjan> anyone has an idea to parse a date like this:
[20:32:32] <Arjan> Tue, 11 Dec 2012 23:36:07 +0000
[20:37:27] <Protagores> RFC-2822 that is?
[20:37:50] <Arjan> I guess yes
[20:38:26] <Arjan> ah here it is https://github.com/lkiesow/erlang-datetime
[20:38:55] <Protagores> just wanted to post that second ago :P
[20:39:04] <Protagores> guess that does it :P
[20:48:39] Protagores leaves the room
[22:32:54] Arjan leaves the room
[22:49:55] andreas.stenius leaves the room
Powered by ejabberd Powered by Erlang Valid XHTML 1.0 Transitional Valid CSS!