Interphase Corp. - V/ATM 5215 - RMF - Document No. UGRMF45-000, Rev. C LinkLevel Driver for the Interphase 5215 V/ATM Hewlett Packard 742/743, HP/RT Release 2.X Last Change, 23apr96, danny waldron ** DESCRIPTION This driver provides both a character-based read/write kernel interface, as well as a sockets-based TCP/IP interface for the Interphase 5215 V/ATM controller. Address discovery (ARP) is handled differently than in older LAN technologies. The current driver provides a direct mechanism to associate host addresses with VC's in a local table, and an application program named `pvcarp' to manipulate that table. In addition, ATM provides rate queues to manage the rate at which data is transmitted, which can be adjusted from a full OC-3 rate of 155 Mb/sec down to the kilobit per second range. This `pvcarp' program has options with which to manipulate the rate queues. There is a special Multi-CPU installation section below. RAWC: Support for Raw Cell reads and writes has been added. This will require an 5215 firmware greater than A04. The The raw cell interface can be tested in a very rudimentary fashion by installing a loopback cable and running: ./ixread -r vc# cnt for receives on one window and on the other enter ./ixwrite -r vc# cnt for the sends Where -r indicates raw mode (AAL0), vc# is the vc and cnt is the number of cells desired. READ/WRITE: The same utility for the raw interface can be used to test the other path through the read/write interface by running the same test sequence without the -r option. In this mode AAL5 frames will be sent and received. CONFIGURATION: You need to insure that the shortio address in the structure in ixrtinfo.c does not have a conflict. The ixrtinfo.c file holds configurable parameters for the controller system operation. The default is a 2K MTU size which correlates to 340 receive ring elements and 160 transmit elements. The basic algorythm is (MTU * # of rcvs) + (MTU * # of xmits) < 1MB. ** INSTALLATION * Before starting ... Before starting the installation, you will need the following: a1) a file named SX00214-A03, which contains the driver image in 'update' format, or a2) a DDS tape labeled SM00013-A00, which contains the driver image in 'update' format, and b) an HP742 or HP743 target system. * Installing the driver in the kernel The following steps occur on the HPUX cross development Unix system that hosts the development sytem for HP/RT. 0) Insert the tape into the tape drive, if you are installing from tape. 1) Run '/etc/update'. If you are unfamiliar with how to use the update utility, refer to the HP man page or manual for instructions. 2) Choose the "Change Source or Destination" option. 3) Choose the "From Tape Device to Local System" menu option from the 'Change Source or Destination' menu. 4) Modify the source field if necessary to reflect either the tape device or the full path name of the file you are installing from. 5) Change the Destination Directory field to reflect the root of your HP/RT tree (usually /HP-RT). Press the F4 key (or equivalent) to finish that screen and return to the main menu. 6) Choose the "Select All Filesets on the Source Media" option, followed by "Start Loading Now". You will then answer 'y' or 'n' as to whether or not you are ready to actually load the software. When the installation is completed, you will be returned to the system prompt. 7) If this is a completely new installation (i.e., you are installing this on an HP/RT system that has never had an Interphase V/ATM 5215 previously installed), you will need to edit the system configuration file. This is named /HP-RT/etc/conf/cfg/CONFIG.TBL (assuming /HP-RT is your HP/RT file system root) and needs to have the line I:iphase.cfg appended to the end. Use your favorite text editor to add these lines. If this is *not* a new installation, you may need to edit CONFIG.TBL and take out the following two SWSM declarations: O:shio:A16+SUP_DATA_ACCESS:0:0xffff:64:SYSTEM_ONLY O:cpu0mem:A32+DATA_ACCESS:0x10B80000:0x80000:0::0666 They used to be added to CONFIG.TBL, but the proper method is to place them in the .cfg file, which is where they now are. If you have these two entries, remove them from CONFIG.TBL. If you wish to change the VME address space being used by the controlling CPU, you will need to edit the iphase.cfg file and set the SWSM address to reflect the correct address range. Also be sure the CPU id is set to reflect the correct CPU number of your controlling CPU. If /HP-RT/etc/conf/cfg/iphase.cfg already exists, you will need to add the line I:ixrt.cfg to the end of the file. If it does not exist do the following copy. cp /HP-RT/etc/conf/cfg/iphase.cfg.5215 /HP-RT/etc/conf/cfg/iphase.cfg 8) You should review the info file to be sure the info matches your configuration. The appropriate file is: /HP-RT/etc/conf/info/ixrtinfo.c This is where you set your interrupt vector value, controlling CPU number, etc. struct ixrtdrv_info ixrtinfo[MAX_BOARDS] = { {VME_3_VEC, /* system int level vector info board #1*/ 1, /* system int info */ 0, /* cpu number */ 3, /* interrup level for driver, must be the same as sys */ SHIO_0, /* shio address */ SDH_OFF, /* sdh flag 0=off 1=on */ IX_SRMTU, /* small rx buffer mtu size */ IX_LRMTU, /* large rx buffer mtu size */ IX_RRMTU, /* raw rx buffer mtu size */ IX_TRMTU, /* tx buffer mtu size */ IX_NUMSRX, /* number of small rx elements to allocate */ IX_NUMLRX, /* number of large rx elements to allocate */ IX_NUMRRX, /* number of raw rx elements to allocate */ IX_NUMRX, /* number of receive elements to allocate */ IX_NUMLTX, /* number of lo-pri tx elements to allocate */ IX_NUMHTX, /* number of hi-pri tx elements to allocate */ IX_NUMTX, /* number of transmit elements to allocate */ IX_VME_BURST, /* packet data vme burst rate */ IX_INTR_CVAL, /* interrupt timer field value for command chnl */ IX_INTR_DVAL, /* interrupt timer field value for data chnl */ IFQMAXLEN, /* tcp queue lenght */ IF_TCP_TXSPACE, /* tcp tx buffer size */ IF_TCP_RXSPACE}, /* tcp rx buffer size */ {VME_2_VEC, /* system int level vector info board #2*/ 1, /* system int info */ 0, /* cpu number */ 2, /* interrup level for driver, must be the same as sys */ SHIO_1, /* shio address */ SDH_OFF, /* sdh flag 0=off 1=on */ IX_SRMTU, /* small rx buffer mtu size */ IX_LRMTU, /* large rx buffer mtu size */ IX_RRMTU, /* raw rx buffer mtu size */ IX_TRMTU, /* tx buffer mtu size */ IX_NUMSRX, /* number of small rx elements to allocate */ IX_NUMLRX, /* number of large rx elements to allocate */ IX_NUMRRX, /* number of raw rx elements to allocate */ IX_NUMRX, /* number of receive elements to allocate */ IX_NUMLTX, /* number of lo-pri tx elements to allocate */ IX_NUMHTX, /* number of hi-pri tx elements to allocate */ IX_NUMTX, /* number of transmit elements to allocate */ IX_VME_BURST, /* packet data vme burst rate */ IX_INTR_CVAL, /* interrupt timer field value for command chnl */ IX_INTR_DVAL, /* interrupt timer field value for data chnl */ IFQMAXLEN, /* tcp queue lenght */ IF_TCP_TXSPACE, /* tcp tx buffer size */ IF_TCP_RXSPACE}, /* tcp rx buffer size */ } The defines for each of the entries is in ixrtinfo.c. Be sure to determine whether you want the board to be running in SONET or SDH mode. Set the entry accordingly. By default (a value of 0, SDH_OFF), the board will be in SONET mode. If you desire SDH (European) mode change the entry to SDH_ON. 9) The installation will load a few files into the following directory: /HP-RT/usr/local/5215 Change to that directory. You may want to review the contents of 'Makefile' before making a new kernel. The relevant variables are: OS What HP-RT version you're using HPRTroot This is the root of your HP/RT tree KERNEL Refers to name of kernel build under the $HPRTroot directory. CPUVER s743 for a 743rt, or s740 for a 742rt CPU. If you are running HP-RT 2.0.3 or earlier, be sure to set OS to 203. If you are running HP-RT 2.1 or later, be sure it is set to 210. When you are ready to build a new kernel with the 5215 driver included, simply type 'make 743' for a 743 CPU, or 'make 742' for a 742 CPU. * Verifying operation Once the kernel has been built, install the 5215 into the system with the 742 or 743 and boot the kernel to verify that your system has been successfully built and configured. From the 742, do the following: 10) Boot HP/RT with the new driver installed, using the command `rtboot -r ramdisk -iC0 lan()/HP-RT/etc/conf/hp-rt.install' 11) If the driver successfully finds and configures the controller, it will print a status message on the console during system initialization. 12) Boot two controllers with the 5215, and connect them via fiber (preferably without going through a switch: it makes things easier to debug). Verify that the link status light turns on for both controllers. 13) After NFS mounting the directory containing the driver and utility programs, verify that the controllers are connected and can pass data by entering `ttcp -r' (to set up for receives) on one machine, and `ttcp -s' (to set up for sends) 14) To test with a single controller, you can use a loopback cable: connect the two fiber ports on a single controller together with one fiber. Boot the system and verify that the link light is ON. Then, login to the HPRT system on two different screens. In one, enter `ixread vc# cnt' and in the other, enter `ixwrite vc# cnt' If the systems are connected correctly, you will see some messages which will indicate that the test ran successfully. * Configuring the network device Once the controller is installed and configured, network (TCP/IP) operation can be initialized and tested with the following steps: 15) Add the IP address and associated VC's of the foreign hosts to be connected with the command `pvcarp -s HOSTNAME -c VC -a' where HOSTNAME is the address of the connected host, and VC is the virtual circuit number to be used when transmitting frames to that host. See the pvcarp man page for other options such as setting a rate for a rate queue. 16) Set the kernel's network configuration with the command: `ifconfig ix0 YOURHOSTADDRESS up' where YOURHOSTADDRESS is the IP address for the system being configured, either expressed in dotted form (128.0.0.0) or as a hostname contained in the /etc/hosts file. 17) After configuring the hosts, verify connectivity with the ping command. 18) An option has been added to pvcarp to get the 5215 Host Statistics Block and other associated information from the driver via an ioctl. From the command line, pvcarp -l 55 , will give the following output: # pvcarp -l 55 5215 Host Statistics Block ctlrsend 5 5 1 ctlrrcv 0 0 0 datachnl0 0 0 0 datachnl1 340 0 0 datachnl2 0 0 0 datachnl3 0 0 0 datachnl4 0 0 0 vme intrs 1 1 rxexc no errors 0 rxexc out of seq COM 0 rxexc out of seq EOM 0 rxexc no bufs small 0 rxexc no bufs large 0 rxexc invalid VCI 0 rxexc invalid VPI 0 RX IF_QUEUE FULL 0 RX/INTR PEAK 0 TCP MAXQLEN 730 Debug level successfully set to 55 ** Multi-CPU Installation Instructions For this section, please note the following terms: Slot1 VME Controller: this 742/743 sits in the first slot in the chassis, and provides some essential hardware support for the bus. Although noted as the VME slot ONE controller, this will be the HP/RT CPU number ZERO. SlotN VME Controller: this 742/743 sits in any slot other than the slot one location, and will be noted as the HP/RT CPU number ONE. HP/RT CPU Number: as noted above, the Slot1 VME controller will be the HP/RT CPU number ZERO, while the SlotN VME controller will be the HP/RT CPU number ONE. To install the 5215 in a Multi-CPU configuration, please use the following steps: 1. Set the jumper on the second 742 controller. There are switches that are described in the system documentation with the 742 that indicate how to specify that the controller will or will not be used for Slot1 system functions. Adjust these accordingly. 2. Install the controllers in the chassis. The configuration that has been tested is as follows: +---+---+---+---+ | S | S | 5 | 5 | | l | l | 2 | 2 | | o | o | 1 | 1 | | t | t | 5 | 5 | | 1 | N | | | | | | 6 | 8 | | 7 | 7 | 0 | 0 | | 4 | 4 | 0 | 0 | | 2 | 2 | 0 | 0 | +---+---+---+---+ Both of the 742's are side by side, followed by the 5215's. Assignment of the 5215's to different HP/RT cpu's is as follows: the lower shortio address controller will be controlled by CPU ZERO. In the above illustration, the 5215 at address 0x6000 will be controller by CPU ZERO (or the Slot 1 controller). The 5215 at address 0x8000 will be managed by CPU ONE (or the SlotN controller). This is not affected by the physical placement of the 5215's in the card cage. 3. Modify system files. You will need to modify the file named `/HP-RT/usr/include/machine/sysdev.h' to assign particular interrupts to the appropriate CPU. The tested configuration had the following lines modified: -------------------- /* Interrupts from each of the 7 VME interrupt levels can be handled by different cpus. Here we define the cpu number which is to handle each interrupt level. */ #define VME_INT_1_CPU 1 #define VME_INT_2_CPU 1 #define VME_INT_3_CPU 0 #define VME_INT_4_CPU 0 #define VME_INT_5_CPU 0 #define VME_INT_6_CPU 0 #define VME_INT_7_CPU 0 -------------------- where VME interrupt levels 1 and 2 are handled by CPU ONE (the Slot N controller), and interrupt levels 3 through 7 are handled by CPU ZERO (the Slot1 controller). 4. Modify driver files. Depending on your configuration, you may need to modify the driver configuration file named `ixrtinfo.c', which contains the following structure: -------------------- /* * NOTE: Make sure the VME interrupt level matches that specified * in the intvec portion (i.e.: VME_3_VEC needs to have 3 specified, * as shown above in the Installation section. */ -------------------- Make sure that the interrupt levels are correct, and accomodate any other controllers installed in the system, and match the CPU assignments described above. 5. Build a kernel. Now that these files are modified, you need to build a new kernel. 6. Boot the Slot 1 CPU. To boot the Slot 1 CPU, the following line has been used: ISL> rtboot -r ramdisk -iC0 lan()/HP-RT/etc/conf/hp-rt.atm which sets the CPU number to ZERO. 7. Boot the Slot N CPU. To boot the Slot N CPU, use the following line: ISL> rtboot -r ramdisk -iC1 lan()/HP-RT/etc/conf/hp-rt.atm 8. Verify. When the systems come up, they will print out a variety of messages that should be helpful in determining that both CPU's saw the correct controller. Use the `ixread/ixwrite' programs with the controllers interconnected to verify operation. ** FILES The following files make up this distribution: README This file swdoc.ps PostScript document describing the interface Makefile Makefile to compile, build, and install ixrt.cfg 5215 Config file for building device special files ixrtinfo.h 5215 Info Header file ixrtinfo.c 5215 Info Source file ixrtdrv.o 5215 driver ixstat Statistics ixread Test program ixwrite Test program ttcp.c TCP/IP test program pvcarp Application program to manage PVC ARP table pvcarp.1 Application program to manage PVC ARP table man page ** BUGS, EXCEPTIONS, ANOMALIES and MORE THINGS TO DO There is a known hardware errata stating that the first two bytes of a raw cell payload may be corrupted when running AAL0 and non-AAL0 traffic at the same time on the raw interface. Add an ioctl flag to the setvcopt to allow turning the VC off/on at the wire. ixread and ixwrite do not support the -r option.