Web lists-archives.com

Re: long I/O delays when strace is running




Daniel Santos wrote:
Well, waiting for GNU/Linux tests to run, so I had a little more time to play
with this.

On 04/23/2017 02:42 AM, Mark Geisert wrote:
Daniel Santos wrote:
Well thank you, I wish I had read this earlier. I've been trying to debug (with
gdb) strace (following children) and now I know why the debugger is blowing
through breakpoints, because I may be debugging strace, but strace debugging
it's own child.  I added a _pinfo::dump () member function that I'm calling from
pinfo::init () after the test "if (!created && !(flag & PID_NEW))" and it's
crashing because for some processes, the entire _pinfo struct isn't mapped in,
just the first page.  Of course, I'm a newbie in this sphere, so maybe I've
missed something and there's sometimes a reason for this?  (The first part of
the struct looks normal.) Otherwise, that would be another odd flaw.

I don't know enough about pinfo.h and pinfo.cc to say anything definite on this.

Well here's one mystery solved.  I modified my dump() function to first dump the
first page of the struct and then the second, so that when it crashes I can see
the contents of the first page at least:

_pinfo 0x30000 {
   pid 2044,
   process_state 0x00004001,
   ppid 2044,
   exitcode 0
   cygstarted 0,
   dwProcessId 0x00000000,
   progname "",

The process_state's bit 0x00004000 happens to be PID_EXECED, so this is
intentionally shorter because there's no more information.

As for the strange _pinfo object that causes the delay, this appears to be an
error in the process database because the windows PID is a duplicate of another
process entry. (I modified ps.exe to output the _pinfo::process_state field.)

$ strace --trace-children --output=/tmp/ps.log --mask=all ps -el
       PID    PPID    PGID     WINPID   TTY         UID STIME  STATE COMMAND
       948    1788     948       1772  pty0      197608 00:04:22 000000c1
/usr/bin/bash
      2720       1    2720       2720  ?             18 00:02:42 00000041
/usr/bin/cygrunsrv
       860       0       0        860  pty0           0 00:04:30 00000001
/usr/bin/cygrunsrv
       984       1    2168        984  pty0      197608 00:04:30 00000041
/usr/bin/ps
      2376    1628    2376       2612  ?         197610 00:02:45 000000c1
/usr/sbin/sshd
      2524    2720    2524       1404  ?             18 00:02:42 000000c1
/usr/sbin/cygserver
      2168     948    2168        860  pty0      197608 00:04:30 00000061
/usr/bin/strace
      1788    2376    1788       2788  ?         197610 00:04:22 000000c1
/usr/sbin/sshd
      1628       1    1628       1628  ?         197610 00:02:45 00000041
/usr/bin/cygrunsrv

The entry that causes the delay is the one with a zero PPID and PGID and the
progname field is wrong --  It isn't "cygrunsrv".  It always has the same
progname name of the previous process.  Notice that the WINPID field has the
same value as the PID of the strace process that's real (2168).

So my guess is that this object shouldn't even be there.

Excellent debugging work! I'm inclined to agree with your last point. I'm poring over pinfo.cc as well as dcrt0.cc, which is the Cygwin DLL init code. The latter talks about special cases if the DLL is runtime loaded (like strace does) vs link-time loaded (like Cygwin apps do). May take me a while to respond.

..mark


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple