Linux Workstation Setup as uTCA Crate Gateway

These notes describe the setup of a mu2etest.fnal.gov linux workstation so that it can be used as a gateway to the VadaTech uTCA crate.  It's primay purpose is to isolate the uTCA crate from the Fermilab network.  It also runs the VadaTech ScorpionWare software to manage the crate, as well as the Xilinx software to configure the FPGA on AMC502 module. 

Note: some of the links in this document can only be accessed on the Fermilab netowrk or via a VPN.

Contents



Linux Installation and Configuration

We started with a clean install of Fermi Scientific Linux 6.7, downloaded from here.  We burned an image of the netinstall CD, and booted from that and configured it as a workstation.  We created a user called "mu2e" and used "visudo" to give this user sudo root privileges (duplicate "root" line for "mu2e" in the file). 

The server software is installed with:
> sudo yum install sshd-server               (needed to ssh in)

Download the latest krb5.conf file from here and copy it to /etc/.

In order to SSH in, the station needs a static IP.  Request one here with the correct subnet.  The confirmation email will include details about NETMASK, gateway, and DNS server.   Right-click on the network icon, select "Edit connection", to change the eth0 connection to "Manual" and enter these values.  They will not take effect until the station is rebooted. 

In order to log in with SSH, you need to get a host key.  Go to the service-now site.  Select "Request New/Renew Accounts"->"Additional Kerberos Items"->"Host and FTP Principals".  Enter node information.  You'll receive a one-time login to generate the host key, along with detailed instructions.  This will add the host key to the key table. 

To log in, create user accounts, and add the correct principle or principles into a .k5login file in the home directory.  Remeember case matters (e.g. prebys@FNAL.GOV).  If the username matches the name on the kerbeos principle, you'll be able to login with your kerberos credentials. 

Some access permissions need to be changed so that the software can be used by regular users.  Specifically,
A second PCIe ethernet card was installed.  To configure it, right-click on the network icon, select "Edit Conections", select eth1.  Set it to "Manual" with
Additional users can be added from a remote login with the
> system-config-users utility

uTCA Crate Setup

The configuration and setup of the crate itself is described here.

Additional Software Installation

Minicom Program

This program will allow the serial port to communicate with the uTCA RS-232 link.  Install with
> sudo yum install minicom

The computer currently has a single, built-in RS-232 port (dev /dev/ttyS0) and a 4-port USB->RS-232 hub (/dev/ttyUSB0->/dev/USB1).  They are currently mapped as follows:
The ports are made world writable, as described above.

A configuration file /ect/minirc.ttyname must be made for each device that is used.  This can be done with, eg
> sudo minicom -o -s ttyUSB0
or by creating/copying a file with that name and editing it.  Once it is created, the port is accessed with
> minicom -o ttyname      (the -o flag prevents the modem initialization string from being sent)
The file "minirc.dfl" has the default configuration, which is used if no "ttyname" is specified.  It currently points at "ttyUSB0".

The files

Exit with CNTL-a q

If it has trouble communicating, try doing CNTL-a x to exit and send a reset string to the serial device.

Always remember to exit the program properly.  Otherwise, it will leave files named "/var/lock/LCK..*" that will prevent the program from running.  If you get the error "operation not permitted".  Go in and delete these files by hand.

The files /etc/profile.d/uTCA.sh and /etc/profile.d/uTCA.csh have the following aliases defined, related to minicom

alias minicom='minicom -o'
alias rs232='minicom -o'
alias minicomA='minicom -o ttyUSB0'     (remove '=' in uTCA.csh)
alias minicomB='minicom -o ttyUSB1'     (remove '=' in uTCA.csh)


Vivado Xilinx Software

Go to this page and then go to the "Vivado HLX" web installer page.  You'll need to create an account if you don't have one.  Once you download the file, do
> chmod a+x (filename).bin
> ./(filename).bin

Select the "Design Suite" option during the install.  The full default installation takes over 30 Gbytes, but this an be reduced dramatially be unchecking the "Ultra" and "Ultra+"  design packages. 

For some reason, you won't be able to install the USB drivers at this point.  Follow the instructions here, except that the "install_drivers" file is located in the directory:
/opt/Xilinx//Vivado/2016.2/data/xicom/cable_drivers/lin64/install_script/install_drivers/  
(use correct version)

Add the files vivado.sh and vivado.csh to the /etc/profile.d/ directory to set up both the path and the environmental variables for the Fermi license server.

ScorpionWare uTCA System Management Software

Go here for the software (contact Eric Prebys for the username and password).  Select the latest version directory, and download and install the CentOS 64-bit version.  The executable will be installed in the /usr/local/bin/ directory and is started with
> sysmanager

The first time it's run, select "New->Carrier".  Give it the name "uTCA" and set the IP address to 192.168.16.1. 

To launch the instance, double-click on this name and just hit "OK" at the username/password dialog. 

Backup Software

Installed 'fwbackups' software by following the instructions here.

ROOT

Installing ROOT on Scientific Linux can be a little problematic because of version mismatches with gcc.  For this reason, we set it up with a remote repositiory using the CERN VM file system (CVMFS).  This will cache any necessary files remotely so that it will run properly. 

Instructions for setting up CVMFS can be found here.

Once it's set up, you can enable ROOT by typing:
> source /cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh
> ups list -aK+ root  (options, to show available versions)
> setup  root v5_34_32 -q e9:prof

It's a bad idea to put this setup in the login scripts, because it can hang if there are network problems.  On mu2etest, the setup has been aliased to 'setup_root' by putting the appropriate root.csh and root.sh files in /etc/profile.d

scopedaq (scope data aquisition)

The crate is currently connected to a Tektronix TDS3054B oscilloscope via the same ethernet interface used to connect to the uTCA crate.  A gigabit ethernet router has been added to enable this.  The scope has been configured to have an IP of 192.168.40.3.   The DAQ software is found in ~mu2e/scopedaq_tds3000.  The latest version can be found archived here.

Once it's unpacked, typing "make" will build the 'scopedaq' executable.  The scope is then configured by editing the 'scopedaq.cfg' file, in accordance with the TDS3000 programming manual. 

The program expects the environment variable VXI_IP to be set, and this is done by putting scopedaq.sh and scopedaq.csh in /etc/profile.d

Scope-Analysis Software


This is a set of root analysis tools to analyze data taken with the TDS3000 scope.  The latest version of the code is here, and its usage is described in the README file. 





(Last modified March 20, 2017 by Eric Prebys).