[MBDyn-users] Socket communication overhead

Richard Crozier r.crozier at ed.ac.uk
Sat Aug 19 16:47:34 CEST 2017


On 03/07/17 23:05, Richard Crozier wrote:
> 
>>>
>>> The local sockets version is virtually instantaneous, but the inet 
>>> socket version takes 23s. Do you get the same result?
>>
>> Indeed I observe similar results; I'll have a closer look at this. 
>> Thanks, p.
>>
>>
> 
> Interestingly I have just got a windows build working with socket 
> support (will send more info about this in the next few days), and it 
> does not have the slowdown for TCP connections. I wonder if a TCP socket 
> connection between two different linux machines would also show this 
> slowdown.
> 
> Richard
>
After looking into this further myself (and asking on 
stackoverflow.com), I discovered that the slowdown can be avoided by 
disabling Nagle's Algorithm (which optimises buffering of socket data, 
but is not applied to local sockets, hence the performance difference). 
The MBDyn communication pattern (one command as uint8_t followed by data 
is apparently not suited to the application of Nagle's algorithm). I did 
this by putting the following snippet in mbdyn_make_inet_socket_type 
from sock.c.

/* disable Nagle's algorithm */
int flag = 1;
int result = setsockopt(*sock,            /* socket affected */
                    IPPROTO_TCP,     /* set option at TCP level */
                    TCP_NODELAY,     /* name of option */
                    (char *) &flag,  /* the cast is historical cruft */
                    sizeof(int));    /* length of option value */


references:

https://stackoverflow.com/questions/45765712/should-i-expect-unix-socket-to-be-50x-faster-than-local-loopback-tcp

https://stackoverflow.com/questions/17842406/how-would-one-disable-nagles-algorithm-in-linux

https://stackoverflow.com/questions/1045530/how-to-set-tcp-nodelay-on-bsd-socket-on-solaris

To see if I could improve performance on a local machine, I also 
implemented a shared memory communicator using boost::interprocess. This 
works, but disappointingly wasn't that different in speed than local 
sockets. The work is in the hg repo I mentioned previously:

https://bitbucket.org/crobar/mbdyn

At the same time I moved the socket communicator to it's own file 
extsocket.c.

I also created several test scripts, based on an example from the 
website for testing things on linux and windows. I've attached these in 
case they are useful. The .bat files are windows batch files. Some paths 
need to be set at the top of the scripts to match your local setup.

Best regards,

Richard






-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: sharedmem.bat
Type: application/x-msdos-program
Size: 2236 bytes
Desc: not available
URL: <http://mail.mbdyn.org/pipermail/mbdyn-users/attachments/20170819/fed10c3c/attachment-0002.bat>
-------------- next part --------------

begin: data;
  integrator: initial value;
end: data;

begin: initial value;
  initial time: 0.;
  # final time: .001;
  final time: 0.1;
  time step: 1e-3;

  tolerance: 1e-9;
  max iterations: 10;

  #derivatives tolerance: 1;
  derivatives max iterations: 10;
  #derivatives coefficient: 1;

  threads: disable;

  #output: iterations;
  #output: residual;
end: initial value;

begin: control data;
  structural nodes:
    +1
    +1
  ;
  joints:
    +1  # clamp
  ;
  rigid bodies:
    +1
  ;
  forces:
    +1  # extnodal
  ;

  default orientation: orientation matrix;

end: control data;

begin: nodes;
  structural: 0, static,
    null,
    eye,
    null,
    null;
  structural: 1, dynamic,
    null,
    eye,
    null,
    null;
end: nodes;

begin: elements;

  joint: 0, clamp, 0, node, node;

  body: 1, 1,
    1.,
    null,
    eye;

  force: 10, external structural,  # force from external program
    shared memory,                 # communicator
      create, yes,
      name, "shared_mem_test_shmName",
      coupling,
        tight,
    2,                      # number of nodes to receive forces
      0, offset, null,      # first node
      1, offset, null;      # second node

end: elements;

# vim:ft=mbd
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sharedmem.sh
Type: application/x-shellscript
Size: 1548 bytes
Desc: not available
URL: <http://mail.mbdyn.org/pipermail/mbdyn-users/attachments/20170819/fed10c3c/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: socket_inet.bat
Type: application/x-msdos-program
Size: 2221 bytes
Desc: not available
URL: <http://mail.mbdyn.org/pipermail/mbdyn-users/attachments/20170819/fed10c3c/attachment-0003.bat>
-------------- next part --------------
begin: data;
  integrator: initial value;
end: data;

begin: initial value;
  initial time: 0.;
  # final time: .001;
  final time: 0.1;
  time step: 1e-3;

  tolerance: 1e-9;
  max iterations: 10;

  #derivatives tolerance: 1;
  derivatives max iterations: 10;
  #derivatives coefficient: 1;

  threads: disable;

  #output: iterations;
  #output: residual;
end: initial value;

begin: control data;
  structural nodes:
    +1
    +1
  ;
  joints:
    +1  # clamp
  ;
  rigid bodies:
    +1
  ;
  forces:
    +1  # extnodal
  ;
end: control data;

begin: nodes;
  structural: 0, static,
    null,
    eye,
    null,
    null;
  structural: 1, dynamic,
    null,
    eye,
    null,
    null;
end: nodes;

begin: elements;

  joint: 0, clamp, 0, node, node;

  body: 1, 1,
    1.,
    null,
    eye;

  force: 10, external structural,  # force from external program
    socket,                 # communicator (a socket in this example)
      create, yes,
      port, 5500, host, "127.0.0.1",
      no signal,
      coupling,
        tight,
    2,                      # number of nodes to receive forces
      0, offset, null,      # first node
      1, offset, null;      # second node

end: elements;

# vim:ft=mbd
-------------- next part --------------
A non-text attachment was scrubbed...
Name: socket_inet.sh
Type: application/x-shellscript
Size: 1495 bytes
Desc: not available
URL: <http://mail.mbdyn.org/pipermail/mbdyn-users/attachments/20170819/fed10c3c/attachment-0004.bin>
-------------- next part --------------
# $Header: /var/cvs/mbdyn/mbdyn/mbdyn-1.0/tests/forces/strext/socket/socket.mbd,v 1.1 2010/03/31 20:08:04 masarati Exp $

begin: data;
  integrator: initial value;
end: data;

begin: initial value;
  initial time: 0.;
  # final time: .001;
  final time: 0.1;
  time step: 1e-3;

  tolerance: 1e-9;
  max iterations: 10;

  #derivatives tolerance: 1;
  derivatives max iterations: 10;
  #derivatives coefficient: 1;

  threads: disable;

  #output: iterations;
  #output: residual;
end: initial value;

begin: control data;
  structural nodes:
    +1
    +1
  ;
  joints:
    +1  # clamp
  ;
  rigid bodies:
    +1
  ;
  forces:
    +1  # extnodal
  ;
end: control data;

begin: nodes;
  structural: 0, static,
    null,
    eye,
    null,
    null;
  structural: 1, dynamic,
    null,
    eye,
    null,
    null;
end: nodes;

begin: elements;

  joint: 0, clamp, 0, node, node;

  body: 1, 1,
    1.,
    null,
    eye;

  force: 10, external structural,  # force from external program
    socket,                 # communicator (a socket in this example)
      create, yes,
      path, "/tmp/mbdyn.sock",
      no signal,
      coupling,
        tight,
    2,                      # number of nodes to receive forces
      0, offset, null,      # first node
      1, offset, null;      # second node

end: elements;

# vim:ft=mbd
-------------- next part --------------
A non-text attachment was scrubbed...
Name: socket_local.sh
Type: application/x-shellscript
Size: 1501 bytes
Desc: not available
URL: <http://mail.mbdyn.org/pipermail/mbdyn-users/attachments/20170819/fed10c3c/attachment-0005.bin>


More information about the MBDyn-users mailing list