Web lists-archives.com

Regression for OCaml introduced by rebase 4.4.4




TL;DR flexlink-compiled DLLs (i.e. ocaml libraries) are broken by the
0x200000000 base address requirement added in rebase 4.4.4. Possible fixes
for this at the bottom.

Commit bfd383 in the rebase sources introduces a new minimum base address
requirement of 0x200000000 for Cygwin64. This is a problem for the correct
operation of the flexdll package and affects ocaml. On a fresh up-to-date
Cygwin64 installation, install the ocaml package:

  $ rebase -i /usr/lib/ocaml/stublibs/*
  /usr/lib/ocaml/stublibs/dllbigarray.so    base 0x000000010000 size
0x00015000 *
  /usr/lib/ocaml/stublibs/dllcamlstr.so     base 0x000000010000 size
0x00014000 *
  /usr/lib/ocaml/stublibs/dllgraphics.so    base 0x000000010000 size
0x00038000 *
  /usr/lib/ocaml/stublibs/dllnums.so        base 0x000000010000 size
0x00011000 *
  /usr/lib/ocaml/stublibs/dllthreads.so     base 0x000000010000 size
0x00025000 *
  /usr/lib/ocaml/stublibs/dllunix.so        base 0x000000010000 size
0x0004c000 *
  /usr/lib/ocaml/stublibs/dllvmthreads.so   base 0x000000010000 size
0x0001f000 *

Here you can see a problem we already know about with flexlink - all
libraries have a base address of 0x10000
(https://github.com/alainfrisch/flexdll/issues/50).

However, this allows you to load libraries dynamically:

  $ ocaml
          OCaml version 4.04.2

  # #load "unix.cma";;
  # #directory "+threads";;
  # #load "threads.cma";;

but not fork (we know about this problem):

  # Unix.fork ();;
        0 [main] ocamlrun 5688 child_info_fork::abort: address space needed
by 'dllunix.so' (0x400000) is already occupied
  Exception: Unix.Unix_error (Unix.EAGAIN, "fork", "").

Now do a rebaseall.

  $ rebase -i /usr/lib/ocaml/stublibs/*
  /usr/lib/ocaml/stublibs/dllvmthreads.so   base 0x0003fec20000 size
0x0001f000
  /usr/lib/ocaml/stublibs/dllunix.so        base 0x0003fec40000 size
0x0004c000
  /usr/lib/ocaml/stublibs/dllthreads.so     base 0x0003fec90000 size
0x00025000
  /usr/lib/ocaml/stublibs/dllnums.so        base 0x0003fecc0000 size
0x00011000
  /usr/lib/ocaml/stublibs/dllgraphics.so    base 0x0003fece0000 size
0x00038000
  /usr/lib/ocaml/stublibs/dllcamlstr.so     base 0x0003fed20000 size
0x00014000
  /usr/lib/ocaml/stublibs/dllbigarray.so    base 0x0003fed40000 size
0x00015000

So forking should now be fine. However:

  $ ocaml
          OCaml version 4.04.2

  # #load "unix.cma";;
  Cannot load required shared library dllunix.
  Reason: /usr/lib/ocaml/stublibs/dllunix.so: flexdll error: cannot relocate
RELOC_REL32, target is too far: 0xfffffffc013d8b5f  0x13d8b5f. 

This is a known problem and fundamental limitation of flexdll (there is no
RELOC_REL64 in COFF). On our CI, we have been using a workaround for the
fork problem at
https://github.com/ocaml/ocaml/blob/trunk/tools/ci-build#L230-L231 but that
no longer works with rebase 4.4.4 because of the new minimum base address.

It was already the case that rebaseall was breaking OCaml DLLs, but now with
4.4.4 they cannot even be fixed by hand, so it's clearly a good moment to
put some effort into this (read as: I'm offering both coding and testing
time!).

For this to work at all, there needs to be some address space below
0x80000000 which DLLs may be permitted to opt-in to using and which rebase
needs to respect. Assuming that's OK, I think something along the following
lines is needed:

 1. We (ab)use either a DLL characteristics flag or a section header flag to
indicate that the DLL needs to be loaded below 0x8000000
 2. The rebase utility warning for base addresses should take that flag into
account (to the point of requiring < 0x80000000 if this new bit is set in
the image)
[2a. While we're changing validation for the image base, it'd be sensible to
add a check that the supplied address is 64K aligned :$]
 3. The flexlink utility should stop using 0x10000 all the time. Probably
the best way to achieve this is if the rebase utility has a flag which
*sets* the new bit so that flexlink calls rebase after compilation to assign
an improved base address to the DLL. On x86, we don't force a given base
address at all - I assume that Cygwin's binutils stuff is already
rebase-aware and produces sensible base addresses for newly-compiled DLLs,
as I don't recall having ever seen the fork conflict problem on x86 builds
of OCaml?

	Comments on the proposed need for some DLLs to occupy memory below
0x80000000 and on the fixes much appreciated, thanks!


David


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple