Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Tree: 0a4a7959b0
Fetching contributors…

Cannot retrieve contributors at this time

200 lines (199 sloc) 15.977 kB
00:32:46 <abdallah> can I host private academic projects on ocamlforge?
02:30:20 <abdallah> can I host private academic projects on ocamlforge?
08:43:26 <pippijn> does ocamlopt do some sort of whole program optimisation?
08:43:34 <pippijn> or inter-modular inlining?
08:43:57 <pippijn> I'm looking at this disassembly: http://paste.xinu.at/pFZuQ/asm
08:44:21 <pippijn> of this function: http://paste.xinu.at/Tplh5/ocaml
08:44:57 <pippijn> and this is Unicode.by_codepoint: http://paste.xinu.at/4vo/ocaml
08:45:51 <pippijn> I don't see a call to Unicode__fun_1234 or whatever
08:46:05 <pippijn> but I see the hashtbl call
08:50:26 <adrien> inlining, yes, if the .cmx files are available
08:50:41 <pippijn> I see
08:50:46 <adrien> whole-program-optimization, no
08:54:33 <pippijn> -rw-r--r-- 1 pippijn pippijn 2.4M Mar 31 16:54 ../_build/libucs/udb_data.cmo
08:54:37 <pippijn> -rw-r--r-- 1 pippijn pippijn 8.4M Mar 31 16:54 ../_build/libucs/udb_data.o
08:54:47 <pippijn> that's some difference
08:54:58 <pippijn> text data bss dec hex filename
08:54:58 <pippijn> 57 5705964 0 5706021 571125 ../_build/libucs/udb_data.o
08:55:11 <pippijn> 5.7MB data
08:55:19 <adrien> yes, bytecode is smaller
08:55:44 <pippijn> less than half
09:25:37 <gildor> hcarty: oasis-db will support 0.3 a week or two after the release of 0.3
09:26:02 <gildor> mrvn: it is not possible to include external file like _tags_oasis in _tags
09:26:37 * gildor 5 min of internet in the WE, sorry I'll probably be AFK in a couple of minutes
09:27:58 <adrien> enjoy your week-end ;-)
09:37:19 <gildor> thx
09:39:54 <zcero> Hello. I'm writing some c stubs and need help to figure out how to manage memory when returning a double* as a bigarray (ocaml should take ownership of the returned array). Any pointers?
10:13:56 <mrvn> gildor: what uses _tags?
10:19:18 <adrien> mrvn: what do you mean? it's ocamlbuild
10:23:29 <mrvn> So option 1: change ocamlbuild to read in _tags_oasis and _tags, option 2: compile _tags from _tags_oasis and _tags_custom
10:24:28 <mrvn> option 3: add an include statement
10:24:35 <adrien> 3: no :-)
10:24:42 <adrien> (it's going to be too ugly ;-) )
10:25:17 <adrien> depending on your uses, you can use -tag, -tags, or -tag-line when calling ocamlbuild
10:25:47 <adrien> also, _currently_, I'd advise to use only one _tags file (or you can use _tags file in subdirectories too)
10:25:53 <adrien> but I agree it causes issues
10:25:57 <adrien> I'd had issues myself
10:26:05 <adrien> I've seen other peoples have issues
10:27:33 <mrvn> And myocamlbuild.ml should include myocamlbuildoasis.ml
10:27:55 <mrvn> or the other way around.
10:29:47 <adrien> there's also the possibility of myocamlbuild.ml.in and _tags.in
10:29:49 <adrien> but
10:30:05 <adrien> I think I prefer not to have that since we have ocaml which is more powerful
10:31:21 <mrvn> I just hate having patch being 10 times as big because of all the generated files.
10:31:39 <adrien> I usually do a separate commit for that
10:31:51 <adrien> works fine enough
11:05:51 <gildor> mrvn: this is issue has been under discussion on oasis-devel ML
11:06:41 <mrvn> any consensus?
11:07:30 <gildor> mrvn: in oasis 0.3 you'll have the option of a standard setup.ml, a setup.ml that weak-depends on oasis (so you can checkout any VCS without needing to install oasis, except if you change _oasis) and an ultra-light setup.ml that need oasis installed but the generate ~0 lines of diff when you change something in _oasis
11:08:04 <gildor> mrvn: I'll suppose you'll go for the ultra-light version
11:08:43 <mrvn> I'm fine with requireing oasis for a git clone.
11:10:12 <gildor> so you'll get something that look likes http://ocsigen.org/darcsweb/?r=lwt;a=headblob;f=/setup.ml
11:10:52 <gildor> e.g 30 lines for setup.ml, just don't forget to run 'oasis setup' when you prepare the tarball
11:12:15 <mrvn> That would be an improvement.
11:21:31 <mrvn> gildor: doesn't solve the _tags issue though does it?
11:21:52 <mrvn> or myocamlbuild.ml
11:21:58 <gildor> mrvn: BTW, this is possible to have myocamlbuildoasis.ml because you have to change the way ocamlbuild compile myocamlbuild.ml (i.e. you have to compile 2 files)
11:22:08 <gildor> mrvn: what is the _tags issue ?
11:22:30 <gildor> the ultra light option will not leave all file blanked
11:23:00 <gildor> i.e. maybe it will create an _tags but its content will be #OASIS_START\#OASIS_STOP
11:23:08 <gildor> nothing in it
11:23:29 <gildor> will not leave -> will leave all file blanked
11:23:44 <mrvn> but when I compile the source it would fill that in or not?
11:24:02 <gildor> off course, otherwise ocamlbuild cannot work
11:24:02 <mrvn> or would it create _build/_tags?
11:24:24 <gildor> no _build/_tags is not in the file scanned by ocamlbuild
11:24:36 <mrvn> then that isn't a solution. It must not alter any file that is tracked in git.
11:24:48 <gildor> but after the run, it will restore it
11:25:36 <gildor> it is a "cp _tags _tags.bak; oasis setup; ocamlbuild; mv _tags.bak _tags"
11:25:56 <mrvn> ugly.
11:26:08 <gildor> well to be more precise it is mv _tags _tags.bak; cp _tags.bak _tags to preserve
11:26:14 <mrvn> and failure prone when the build aborts
11:26:14 <gildor> file property
11:26:36 <gildor> mrvn: I am waiting a better solution/patch that will work without changing ocamlbuild
11:26:47 <mrvn> gildor: why not change ocamlbuild?
11:27:18 <gildor> mrvn: you can, I just won't do it
11:27:53 <mrvn> gildor: Then have a _tags.in and generate _tags from that
11:28:48 <gildor> mrvn: ok, send me a patch
11:29:28 <mrvn> the ocambuild -tags option. Does that expect file names or actual tags?
11:30:31 <gildor> mrvn: heu, -tags doesn't do that at al
11:30:46 <gildor> mrvn: it just add a couple of tags to all target
11:31:23 <gildor> mrvn: -tag is equivalent to a line in _tags
11:31:50 <gildor> ps: don't call it _tags.in, call it _tags.oasis
11:32:19 <mrvn> fiNo, the _tags.in would be the users tags.
11:32:22 <mrvn> -fi
11:32:45 <mrvn> The extunix _tags now contains:
11:32:50 <mrvn> <test/*.ml{,i}>: pkg_bigarray
11:32:50 <gildor> pps: don't drop the default scheme, just check the presence of _tags.oasis and trigger your behavior if it is present
11:32:57 <mrvn> <test/*.ml{,i}>: use_extunixba
11:33:08 <mrvn> But only test/testba.ml needs that.
11:33:35 <gildor> mrvn: you can add <test/*.ml{,i}>: -pkg_bigarray
11:33:40 <mrvn> Shouldn't oasis be more specific with those tags?
11:33:54 <gildor> "test/extunixbe.ml{,i}": pkg_bigarray
11:34:02 <mrvn> gildor: Then test/testba.ml is not getting that
11:34:33 <mrvn> It is not something I should have to manually specify. That is what _oasis is for.
11:34:35 <gildor> mrvn: that is an executable and I have corrected this
11:35:42 <gildor> the current version of oasis is not the one that is set to last forever, this is a tool in evolution
11:36:08 <gildor> this version fits my need but not yours, lets discuss and improve that
11:36:08 <mrvn> gildor: sure. Great if it is already improved. On that note: /usr/bin/oasis: unknown option `--version'.
11:36:35 <gildor> oasis 0.3: 'oasis version'
11:36:40 <gildor> already fixed
11:36:56 <mrvn> Lets hope debian gets a new oasis soon then :)
11:37:29 <gildor> oasis 0.3 is only ~rc3
11:37:50 <mrvn> .oO(experimental)
11:37:55 <gildor> but feel free to join the oasis-devel ML if you want to have a more open discussion
11:38:16 <mrvn> gildor: nothing to discuss. you are already doing what I want. :)
11:38:48 <gildor> mrvn: use the darcs version to send me a patch for the _tags.oasis stuff you want
11:39:59 <mrvn> At the moment I just want to get extunix finished so I can get back to my own stuff.
11:40:06 <gildor> (this file need to contain #OASIS_START/STOP section which will be the replaced part and you can generalize that to all file generated: myocamlbuild.ml, foo.mllib...)
11:40:49 <gildor> mrvn: the more simple solution: move extunixba.ml into src/bigarray/
11:41:13 <gildor> 1 min change and all will work (different library in different directory)
11:42:11 <mrvn> would also avoid the name conflicts between files for standard and bigarray functions.
11:42:22 <gildor> mrvn: indeed
11:43:11 * gildor got to go
11:43:33 <mrvn> got to write bindings for read, writem recvmsg, sendmsg with bigarrays
11:45:01 <avsm> mrvn: Lwt_bytes has bindings for all those, if you're using Lwt
11:45:27 <avsm> mrvn: or are you creating alternative non-string versions of the Unix library or something (that would be useful too)
11:45:40 <mrvn> avsm: I'm not and I'm not convinced lwt does it right.
11:46:17 <avsm> alternatives always good; but i haven't seen any quite as complete as lwt yet
11:46:21 <mrvn> avsm: kind of. Except Unix doesn't have pread/pwrite/recvmsg/sendmsg
11:49:17 <mrvn> args lwt has pread/write that are flavours of popen() and not bindings for pread() and pwrite()
11:52:06 <mrvn> avsm: The lwt read/write bindings don't release the runtime system so they are not thread capable.
11:52:40 <mrvn> So read/write always blocks everything. Totaly unusable imho.
11:53:30 <adrien> I quite like the fact that setup.ml should be complete and in the VCS
11:53:30 <mrvn> I guess lwt assumes it only has to handle non-blocking sockets and pipes and real file IO takes no time.
11:53:53 <adrien> it's a bit annoying for the developer but for people building, it's pretty good
11:54:10 <adrien> actually, it's like with gobject-introspection
11:54:29 <adrien> a number of files get generated but it's actually a nightmare for the packagers and it's not needed
11:54:35 <mrvn> adrien: That is why you include it in tarballs. But in git it is fine to just generate it as needed.
11:54:38 <adrien> it doesn't even make that much sense to have everyone generate them
11:54:52 <adrien> it'll be forgotten at some point, and more often than not
11:54:55 <adrien> or not in sync
11:55:29 <mrvn> adrien: can't be out-of-sync if it is not in git. can't be forgotten if the make tarball target creates them.
11:56:07 <adrien> but with it all the time, a git clone and git pull will bring a working setup.ml and friends
11:56:37 <adrien> if the file should be the same everywhere, why not generate it only once, including in the VCS
11:56:57 <adrien> there not doing so introduces so many new occasions for bugs to appear
11:59:30 <mrvn> adrien: I disagree
12:06:38 <diml> mrvn: lwt handles differently non-blocking sockets and pipes vs the rest
12:07:09 <mrvn> diml: where?
12:08:05 <diml> in the ocaml functions, it checks whether the Lwt_unix.file_descr is blocking or not and choose what to do
12:11:22 <diml> but if you are not using lwt, it is better to write your own stubs that always assume the operation is blocking
12:15:08 <mrvn> diml: from what I can tell on blocking channels ltw creates a job for the read and calls execute_job. Does that fork?
12:16:39 <diml> mrvn: the job is executed in the current thread (blocking all lwt threads if it blocks) or in another preemptive thread
12:16:48 <diml> you can control that locally or globally
12:17:06 <diml> by default it is executed in another thread
12:17:40 <diml> (and there is also a third method, which has better performances, but it is an ugly hack)
12:19:19 <diml> the best solution would be threadlet but it was never included in the kernel...
12:25:53 <mrvn> or do async io.
12:27:16 <diml> yes, but async ios are not implemented on all file systems, they transparently fallback to blocking ios and there is way to tell whether it is supported or not :/
12:27:48 <mrvn> diml: posix async IO uses threads anyway.
12:28:10 <mrvn> And what filesystems under linux don't support the async IO functions?
12:32:50 <diml> i checked that a long time ago, i should have another look
12:33:26 <diml> btw, in my tests, if you increase the buffer size to at least 64KB, you get the same results with or without threads
12:34:39 <mrvn> diml: the difference comes when you fire of 1000 requests. Threads simply stop at 16 or so and put the rest in a queue for later.
12:35:49 <mrvn> And you need to use bigarrays to avoid the extra memcpy() operations needed with strings.
12:37:11 <diml> mrvn: i did tests with sequential reads/writes when everything is cached (the worst case when using threads)
12:38:00 <mrvn> diml: no, the worstcase is random reads that aren't cached but collectively end up being sequential.
12:38:04 <diml> and i did use bigarrays
12:38:31 <mrvn> with threads that causes tons of seeks, with linux aio it can scatter gather them back to sequential.
12:38:34 <diml> mrvn: i mean the worst case for threads vs non threads
12:40:10 <diml> if the call really blocks it is not a problem to use threads, if it doesn't then threads are a big overhead
12:41:37 <mrvn> except when the low number of threads you have causes extra seeks. You get that for p2p stuff where you easily have 100-1000 clients.
12:42:35 <chambart> I imagine that it would be possible to delay the launch of the threads such that you can reorder things a bit
12:42:56 <chambart> but that would increase latency for normal cases
12:43:07 <mrvn> That assumes you have knowledge of the physical layout of the blocks
12:43:21 <mrvn> think raids or fragmented files
12:44:51 <chambart> if you have things that looks sequential, the kernel can helps you a bit for that
12:45:40 <chambart> but when you want to do agressive stuff like that it is probably better to use a mmaped file interface
12:45:58 <mrvn> chambart: verry bad for writing.
12:46:36 <chambart> mrvn: why ?
12:46:52 <mrvn> chambart: The linux aio interface allows you to just send 1000 requests to the kernel and let the kernel split them up, merge or reorder as needed for the physical devices.
12:47:03 <flux> doesn't that happen with mmapping?
12:47:12 <flux> it's not like mmap = synchronous
12:47:25 <mrvn> chambart: When you write 4k to a mmaped file the first byte causes the page to first be read and the modified.
12:47:42 <flux> the benefit of AIO is that yuo know when the write has happened
12:47:45 <mrvn> plus mmap blocks on access if the page isn't present
12:47:45 <chambart> you can do batch write on mmaped files
12:48:09 <mrvn> chambart: how?
12:48:33 <chambart> it is why you should use mincore/madvise to help the kernel ordering reads and avoid blocking
12:49:01 <mrvn> chambart: that doesn't supress the unneccessary read when you write a full page.
12:49:06 <chambart> you should use msync to write
12:49:20 <mrvn> chambart: s/should/must/
12:50:54 <flux> so writing 10 MB to a mmapped device results in 10 MB of reads as well?
12:51:02 <mrvn> flux: yes
12:54:11 <mrvn> catching errors with mmap is also a pain. on read you get a segfault. on write I think msync fails.
13:14:03 <mrvn> Phantom type question. Anyone have an example of a container with phantom types for both the container itself and the objects it contains? So that one can have a hashtbl of const strings or a const hashtbl or mutable records or a const set or mutable hashtbl of const strings and so on.
13:14:21 <mrvn> Something that works recursively.
15:30:51 <hcarty> gildor: Thanks for the update. I'll hold off on trying to push anything to oasis-db using a newer oasis until then.
16:54:43 <Ulrar> Hi, just saw that ** is an operator for pow, but does it exist for int ?
16:58:26 <mrvn> # ( ** );;
16:58:26 <mrvn> - : float -> float -> float = <fun>
16:59:01 <mrvn> should be called **.
16:59:33 <Ulrar> Yeah, so there is no pow operator for int ?
16:59:56 <mrvn> not that I'm aware of. Wouldn't be that usefull given the limited range of int.
17:00:23 <Ulrar> Okay, thanks.
17:00:42 <mrvn> # let ( ** ) x y = int_of_float ((float_of_int x) ** (float_of_int y));;
17:00:42 <mrvn> val ( ** ) : int -> int -> int = <fun>
17:00:49 <mrvn> # 4 ** 4;;
17:00:49 <mrvn> - : int = 256
20:17:11 <Juzor> mrvn: hi, again
Jump to Line
Something went wrong with that request. Please try again.