$Id: README,v 1.20 2001/06/29 22:09:17 randall Exp $

This is the USER deamon version of SCTP. It
uses a small deamon that does the IP transfer. This
deamon MUST be suid to root so it may open a RAW_IP
socket or two.

Once open it accepts IP datagrams tunneled from the
user library via UDP. It will drop any SCTP datagram
on the wire from a registered user. It also will
forward SCTP datagrams to the respective UDP port,
has long as the library has registed with the deamon.
There is no pinging/heartbeating between the deamon
and registered users, so if a user process cores without
de-registering, you may get a situation where the
next UDP user that binds that port may get some
IP messages tunneled back to it. 

To make this a bit more efficent a shared memory segment
would probably be in order between the library and the
deamon, then the UDP could be used has a tickle pipe only
to wake up the deamon. If this was done, one would probably
be best to implement the partial delivery api, or consider
it. This brings me to the list of things not done:

1) We did NOT implement a partial deliver API. This
   was felt un-needed in a user level implemenation, the
   user side will just calloc up as much memory as needed.

2) We did NOT implement the revoking of messges based on
   a time specification in the SEND() primitive. The idea
   of this is to have a "life" associated with a message
   being sent. If it has not been put on the wire before
   the "life" expires, then bounce it back to the user saying
   I could not deliver it in time. This is not implemented and
   would require some minor additions. Right now this implementation
   assign's TSN's when queued. This could be changed to assign
   TSN's later OR a re-numbering scheme could be done so that
   when a datagram is revoked, you would go on down the ordered
   list re-numbering the datagrams. If this is done be sure
   to renumber the ->data pointer inside the chunk as well
   as the structure TSN_seq in the transmit queue.

3) ECN has been implemented but it is un-tested.

4) In general ABORT will carry a error if it is simple and
   fits within a NO DATA type error cause. i.e. the out
   of resources error cause etc. We have added support
   for multiple erro causes AND support for sending op-errs
   bundled with DATA/SACKs etc.

5) No sort of bundling delay is implemented in this implemenation,
   the only bundling that will occur is due to network congestion.
   This bundling can not be turn-off by the ULP.

6) Path MTU discovery is implemented completely but will
   not work on Lynx O/S due to there silly access to
   RAW IP does not allow sending your own IP header.   

7) We support DNS name input, by using a blocking call
   to gethostbyname() this may cause some retrans etc.
   If the name query on the INIT side fails we 
   send a restricted addresses and resend the INIT,
   telling the peer NOT to give us any host name
   parameter.. i.e we support v6/v4 addresses only.
   Also a abort with the appropriate error cause
   'unresovable address' is sent back.

8) We have added limited support of the error cause chunk.
    

Suggested performance improvements:

1) isThereAGap can be optimized out by possibly keeping a flag.
2) buildASack should be optimized.
3) update_alder32 could be done more efficently.
4) keeping the associations in a Hash Table, not a linked list.
5) splitting the outgoing data into two lists, queued and transmitted.
6) Change the communication between deamon and clients to
   used shared memory (and add partial delivery api).

In general this implemenation is fairly complete. It is not
a star performer and with a bit of playing with gprof one
can find some easy things to do to speed things up. 
I have done some of this so it has improved over the
last couple of releases.. but there is always room
to make it better :).

Idealy changing the association lookup to a hash table instead of
a linear search would be a good idea, as well as splitting
the outbound lists into two lists, commited to transport
and inqueue could also improve performance. Here and there
throughout the code are comments that indicate weaknesses.
But I am sure there are many others. Please let me know
of any bugs and other such things you find, I will either
try to fix them or add them to the README :-)

A list of the files and a brief description:

top level directory:

COPYING        - Rules for copying/using this distribution (GPL).
GNUmakeconfig  - The make config file, here you change -g to -O3 etc.
GNUmakefile    - Basic makefile.
GNUmakerules   - more make file goo, normally you don't touch this one.
GNUmaketargets - more make file goo, normally you don't touch this one.
README         - this file.
TODO.list      - My list of things to do, it will grow I am sure :-)
patchlevel.h   - This file is used to keep the major/minor revision level
                 and help make sure you don't apply the wrong patch (if
                 I put out a patch)

apps directory:

GNUmakefile     - Its the standard make file that describes what apps are made.
hashWalk.c      - A test program for testing the hash table.
md5_test.c      - This is a little md5_test I put together to compare it to
                  SHA-1.
patchlevel.h    - Ditto, patchlevel above :)
primelist.c     - A program that generates prime numbers, used to create
                  the header file the Hash table uses.
rtest.c         - A silly program used for testing MD5 performance.
sctpDeamon.c    - This is the main for the IP deamon, it must be SUID to root
                  so it can open a RAW IP socket.
sctp_test_app.c - This is the main loop and timer stuff.
sctprtt.c       - This is a fun little utility that will take a bunch of
                  input RTT times and run Van's and our algo on it to generate
                  RTO's. I used this with Mark Almans help to generate some 
                  nice xgraph's.
sla_1test.c     - First test module so I can input the 1st test cases
                  given in FIP180-1.
sla_2test.c     - Second test, so I don't have to input the entire
                  second test case from FIP180-1, I did it for
                  a while and I hated typing in the long string :-)
sla_test.c      - Third test case generation defined in FIP180-1.
stdintest.c     - A test program used to test the distributor.

baselib directory:

GNUmakefile     - Gnu make file for the library.
HashedTbl.c     - A generic hash table.
HashedTbl.h     - Header file for hash table, lots of info in here.
StateMachine.c  - A state machine that works with the distributor.
StateMachine.h  - The header file for the state machine, comment coming soon :-0
distributor.c   - This is the main Reactor, that implements the reactor model.
distributor.h   - Header file for distributor.. many good comments on use here.
dlist.c         - A doubly linked list.
dlist.h         - The header file for the double linked list, many good
                  comments on use in here.
llist.c         - A singly linked list, same functional signatures as the
                  dlist so transition betweeen them is easier.
llist.h         - The header file for the singly linked list, many good
                  comments in here.
msgDirector.c   - The message re-director 
msgDirector.h   - The header for the re-director. Some comments, basic idea
                  is instead of subscribing 1,000 objects you use this
                  to subscribe ONE object (this guy) that understands
                  (through your code) how to understand a message and find
                  the object you wish.
patchlevel.h    - Ditto, patchlevel above :)
primeList.h     - A wonderous list of hash prime numbers. 
return_status.h - The standard return statuses used by all (or I try to
                  make it all :/)

user directory:

GNUmakefile      - Gnu make file for the library.
c_crc16.c        - This is the code for the Adler32 stuff.
c_crc16.h        - Header file externing the interface.
md5.c            - This is the md5 code, direct untouched from the RFC.
md5.h            - This is the extern for the md5 code.
md5driver.c      - This is the driver that has will drive  MD5 or
                   SHA-1 and generate a signed cookie. Mostly taken right
                   from the RFC.
md5driver.h      - Externs for the driver.
md5global.h      - Global structures for MD5/SHA-1
patchlevel.h     - Ditto, patchlevel above :)
routingAdaptor.c - This module provides a C adaptor for the distributor
                   that will open the read only routing socket to get
                   interface changes and will translate these into
                   add/delete ip requests.
routingAdaptor.c - Header file that defines the external interfaces to
                   the routing adaptor code.
routingAdaptDef.h- A header file defining some things for our routing
                   socket code.
sctp.c           - Main entry point for all user interfaces and some
                   internal things.
sctp.h           - Extern for all accessable functions to the user.
sctpAdaptor.c    - This is the adaptor that will fit the sctp implementation
                   to the framework (distributor). Still need to get notify's
                   being sent as messages.
sctpAdaptor.h    - header file for adaptor.
sctpConstants.h  - Various constants for SCTP.
sctpDeamonUtil.c - This is the work horse part of the sctpDeamon.
sctpDeamonUtil.h - This is the externs for the work horse.
sctpHeader.h     - This file contains all the struct definitions
                   for the wire.
sctpStructs.h    - This is the main internal structures and such used
                   to track associations and queues and such.
sctpaddipext.c   - The code that implements the ADD IP /DEL IP function
sctpaddipext.h   - Header file for ADD/IP.
sctphandleto.c   - This module handles the timeouts.
sctphandleto.h   - externs for timeout mod.
sctpinbound.c    - This handles all inbound DATA. 
sctpinbound.h    - externs for inbound data.
sctpinitiate.c   - This handles init/init-ack and most control messages.
sctpinitiate.h   - externs.
sctpmtu.c        - This is the low level send that generates IP. Also this
                   is where ICMP is handled.
sctpmtu.h        - externs.
sctpoutofblue.c  - This is where out of the blue packets are dealt with.
sctpoutofblue.h  - externs.
sctprelreq.c     - Code that handles the REL-REQ and calls the add ip or
                   whatever else is needed. This will probably go away
                   and be merged into the addip files when the REL-REQ
                   disappears
sctprelreq.h     - REL REQ header file.
sctpsenders.c    - This is where all the sending things come to send data.
                   These always in the end lead to the low level send in
                   sendmtu.c but this is where the last bit structures
                   are formed.
sctpsenders.h    - externs.
sctputil.c       - This is a collection of utilities used through-out the
                   code.
sctputil.h       - externs.
sla1.c           - This file should be called SHA1.c not sla1.. hey I was
                   tired when I did this (in France at ETSI)... It has my
                   home grown SHA-1 algorithms in it.
sla1.h           - externs and such for SHA-1.
userInputModule.c - This is the user interface module for sctp, when someone
                    types on stdin this is where it checks for commands
                    and what is being sent.
userInputModule.h - externs.

FreeBsdKern/net

rtsock.c	  - This is an update for the rtsock (the routing socket)
                    that allows one to have a SOCK_DGRAM type routing
                    socket only that gives you read only access to
                    the routing table. This will allow a non SUID
                    user process to see new addresses and such. Made
                    for 4.2. We also fix it so that rtm_index is
		    filled on adds/del's and changes.

FreeBsdKern/net43

rtsock.c          - ditto for 4.3 (code has not changed).

FreeBsdKern/conf

files             - Kernel make utility list that adds the sctp_usrreq.c to 
                    FreeBSD build list.

FreeBsdKern/netinet
sctp_usrreq.c     - Kernel SCTP that does RAW IP
sctp.h            - header file with SCTP stuff for kernel.
sctp_var.h        - header file with various kernel structures
in.h              - Modified to include the SCTP protocol.
in_proto.c        - Modified to add the proto switch for sctp

FreeBsdKern/netinet6
icmp6.c           - made a function non-static for rate testing.
in6_proto.c       - added protocol 
sctp6_usrreq.c    - V6 version of above
sctp6_var.h       - ditto.


FreeBsdKern/netinet43 - FreeBSD 4.3 variety
sctp_usrreq.c     - Kernel SCTP that does RAW IP
sctp.h            - header file with SCTP stuff for kernel.
sctp_var.h        - header file with various kernel structures
in.h              - Modified to include the SCTP protocol.
in_proto.c        - Modified to add the proto switch for sctp

FreeBsdKern/netinet643 - FreeBSD 4.3 variety
icmp6.c           - made a function non-static for rate testing.
in6_proto.c       - added protocol 
sctp6_usrreq.c    - V6 version of above
sctp6_var.h       - ditto.


LinuxKern/ipv6

raw.c             - This is a substitue raw.c file to allow the
                    deamon to do true IO via ipv6. If you do
                    not put this in place DO NOT use IPV6 on
                    Linux. The raw socket does not support
                    IP_HDRINCL and thus we can't get the full
                    IPV6 header without this change. Not counting
                    the strange things it does to output.


TODO: We probably need to make it so a O/S without IPv6 
can compile the code. Right now if I compile on a solaris
machine without the V6 patches/addon we will blow up big
time. We could put a set of ifdef's in and a dummy
file that contains all the IPv6 structures... but we
may just want to live with it. I think Solaris 2.9 has
all the V6 stuff in it.... so it may be moot by the
time I get around to this item. Other fish to fry now
I need to go dig in to rewrite into base BSD a compelte
SCTP stack..

Some of the patches since 4.0.4


NOTE: You NEED to have readline from GNU installed
      on your machine.

Randall moved the validation of no new addresses
up into INIT in processing. So if a previous
association exists, it verify's no new
addresses.. if new addresses it ABORTS.. except
in the case of COOKIE_WAIT. Here we override the
from address to be where we sent our INIT.

Marco added the command line interface. This may
break some O/S versions.

Randall integrated Bruce Zaho's mod's for Solaris. I hope
the interaction of command line editting and the Solairs
stuff is not to bad :/

minor tweaks and bugs fixed as called attention to.

Fixed the FR algorithm to block FR's after a T3-timeout
until after the Cum-Ack passes the higest point in the
timeout.

Added code to silently discard a packet if an INIT
is bundled with anything else i.e. the implementors
guide.



Some of the patches since 4.0.3


This release includes a tested and fixed P-MTU for both
IPv6 and IPv4. We also include fixes so that link local addresses
will work if you specify the scope variable to indciate which
interface you wish to send to.

The code in the library went through more debugging and changes.
Support for recognizing the ENOHOST ENODEST type errors from
sending are now added. Making it so you no longer need
to wait for the retransmissions to realize the association will
not come up. 

I hope to add in for BSD routing sockets a automatic
hookup to the add/delete address facility. I finished
testing the add/delete and it now functions (after 
a few bug fixes :0)

Various bugs were also found in the handling of the PMTU stuff
for both V4 and V6. These were all fixed so now we correctly
deal with PMTU changes.

Fixes were put in for the use of /dev/random to improve
the quality of random number selection. Special thanks
to Steve Belovin for advice on the best approach to
use.

Still TODO:

  - Validate the ADD/DELETE/SETPRIMARY
  - Routing socket hookup for freebsd to the 
    add/delete/setprimary.
  - srwnd
  - u-sctp
  - updates to add/delete/set for single chunk types
    but I must finish the drafts first :)
  - Removal of the REL-REQ stuff.


Some of the patches since 4.0.2

This release includes a kernalization for FreeBSD. The code
included is for FreeBSD 4.2 Release. If you have something
more current you can always figure out the differences between
in.h (mainly just adding SCTP) ,in_proto.c (the protocol
switch to add sctp), inpcb.c (a new function) and inpcb.h
(proto for new function), and all the ipv6 ones as well.
Most all changes are small and you just need to add the
sctp files for the protocol. 

For those of you on linux wanting ipv6 you need to add
the patch. Linux seems rather brain dead for a number
of reasons:

  - IPv6 raw sockets don't let you at the IPv6 header (without
    my patch). The patch fixes this but then ALL raw sockets
    get the IP header.. be warned.

  - IPv6 raw sockets will not let you specify an address of
    0 and pick the address of the interface. This means the
    braindead IPV6 source addr selection is what must be
    used... ugly. There is a compile switch for this if
    it is ever fixed...


This release (4.0.3) Fixes numerous bugs. Does a MAJOR
update to make it so IPV4/V6 works amongst just a few
things. We also have the kernel support for FreeBSD.

Now Still yet to do that will come in 4.0.4 is to hook
in the BSD routing socket via a independant distributor
module to get ifconfig changes. This will make the protocol
stack suited for automatically adding/subtracting interfaces
has they are ifconfig'd..



Some of the patches since 4.0.1

user/sctpmtu.c
user/sctp.c

support for the FreeBSD kernel extension.

apps/sctpDeamon.c

o getLoweLevelIP():
  - describe what the function does
  - fix misplaced debug outputs

o main():
  - check for the return value of DEAMONinitDebug()

user/sctpDeamonUtil.c

o DEAMONinitDebug():
  - fix a local security hole that allowed an unpriviliged user to overwrite
    any file in the system. See the comments for more information.

user/sctpDeamonUtil.h

o DEAMONinitDebug():
  - change type from void to int.




Updates since Release 3.1.2

Changes added by Jim Bound and Sridhar Gopalaswamy to support
the tru64 and freebsd systems.

Wow, we now have a framework with base lib .. big changes
here. We also have fixed several bugs including the
sctpDeamons spinning forever bug (at least on linux).

We have added the support for the REL-REQ/ACK and 
ADD/DELETE IP address. I have put in some
comments here and there and I have marked for
Phil Conrad and crew the places to add the s-rwnd
feature at. I am sure 4.0.2 will have this :)



Updates since Release 3.1.2 from Version 3.1.1 

- Bug fix in SCTPcomplexCookie() missing CookieAck was not
  being sent.

- Added more sophisticated method to do the Cisco Echo/Resp
  and simple case stuff. This will make the implemenation 
  work in the bakeoff. We now have a flag to turn on the
  new mode or back to the ascii old mode.

- Bug reported in the Cisco stuff in calculating the
  size that goes in the binary structure... typo had did
  a sizeof(sizeof()) ... so the data was sent but the
  size content of the header binary was wrong...



Updates for 09/06/2000 Release Version 3.1.1 from Version 2.2.3

- Major UPGRADE to v13 of the SCTP spec.

- Bug fix in the SHA-1 modules that caused a nasty core.

- Bug fix to the performance stuff that caused a timer race condition
  to make things go wacko

- Enhanced the deamon to send a ABORT back to someone when no one
  is registered for that SCTP port (i.e. OOTB).

- Bug fixes (loads) from the inter-op.

- Abort2() was added to just abort a single association.

- Some user interface features were added, some for
  debugging, some to support a test tool I use.

--

Updates for 5/25/2000 Release Version 2.2.3 from Version 2.2.2

- a LOT of performance enhancements.

- serious bug fix in the way a_rwnd and rwnd were dealt with.

- Bug fixs for Lynx O/S in relation to raw socket stupidity in
  the way Lynx will not allow you to put your own IP header on.

- Fix so that a BSD_4_4 define exists for finding the addresses (again
  Lynx OS driven)

- Updates to make for lynx

- Adds support for sending a Invalid Stream error back to a offender.

- Fixes a bug in SACK generation.

- More subtle bug fixes found by the Telecordia SCTP team.

- Bug fix in the deamon for a large cookie handling.

UPDATES for 5/02/2000 Release Version 2.2.1 from Version 2.1.1

- Additional things put in through-out code to meet conformance
  test considerations.

- Bug fixed - In building SACK's in sctputil.c in size
              calculation and in the network encoding.

- Bug fixed - In the sctputil, the offset was miscalcuated
              of the start of the optional parameters.

- Bug fixed - The main loop had a condition that needed to
              be fixed that could cause one to never exit.

- Bug fixed - More bugs fixed with the above miscalcualtion of
              optional params.

- Bug fixed - The INIT message would allow a TAG of 0 in.

- Bug fixed - Slight performance related bug in handling of 
              building a SACK (not impacting code for
              correctness just performance).

- Bug fixed - sctputil.c had a cloning error bug in
              a re-init case with stream allocations.

UPDATES for 4/14/2000 Release Version 2.1.1 from Version 2.1.0

- Bug fixed - A bug was fixed which caused the test app to core
              after 1/2 hour. This was due to the H-MAC recalculation
              of its random string had a incorrect if in sctp.c

- Bug fixed - A bug was fixed in userInputModule.c where if pingcount
              ever reached -1 it would stop collecting input.

- Bug fixed - A bug was fixed in that if you attempted use other
              than stream 0 and you were not the intiator, the
              process would core, added the missing code to
              sctputil.c when unpacking a cookie to setup the
              proper strmin/strmout structure.

- enhancement - The PMTU raise function was enhanced to have a
                ceiling on its raising of the maximal PMTU to that
                of the largest MTU of all the interfaces. gatherAddresses
                was enhanced to figure out which MTU was largest and save
                that in the SCTP structure. This is used to bound
                MTU calculation in the sctpmtu.c module that raises the MTU.

- Misc chg   - The protocol ID in sctpConstant.h was changed to the official
               SCTP protocol id 132.

Special thanks and kudo's to all the folks who have tested and sent both
fixes and problems in, current list I have is as follows:

Mitch Miers - mmiers@intellinet-tech.com
Hal Rosenstock - hrosenstock@telica.com
Andreas Jungmaier - ajung@exp-math.uni-essen.de
Ashley Clementi - ashley.clementi@ot.com.au
Larry Wong, Sunil Samtani, and Mallesh Kalla (the telecordia SCTP team) - larryw@telecordia.com
Heinz Prantner - hprantner@ss7-link.com
Renee Revis - drrevis@cisco.com
Mario Hebert - Hebert_Mario@hotmail.com)
Jim Bound    - bound@zk3.dec.com
Sridhar Gopalaswamy - Sridhar.Gopalaswamy@compaq.com
Venkat Venkatsubra - venkats@austin.ibm.com (Thanks for the AIX fixes :>)
Marco Molteni - mmolteni@cisco.com
Bruce Zaho - bzaho@cisco.com

Sorry if I missed anyone (please let me know and I will
correct the above).. thanks for all the help!!

Have fun

Randall R. Stewart