; readWord success
  
install Tensorflow from source on Ubuntu Server 18.04 (2020-03-14) cella 2019 Branchable 0 12657
  

================================================================================ ubuntu

==== make a installation USB stick after downloading Ubuntu Server 18.04 non-live version


==== BIOS setting
- F2 key after turning on
- select the USB stick as the first priority when booting


==== install Ubuntu Server 18.04
- select automatic update for Security modules
- yes to almost of selections

$ sudo apt update
$ sudo apt upgrade


==== ssh
$ sudo apt install openssh-server
- if failed after reinstallation, edit ~/.ssh/known_hosts in Mac OS X.
$ sudo ifconfig
- local IP address after inet.


==== wifi connection to WPA2 hidden network with ipTime N100 mini
$ sudo apt install network-manager
$ sudo systemctl start network-manager
$ nmcli d
DEVICE TYPE STATE CONNECTION
wlxx wifi disconnected --
...
$ sudo nmcli device wifi connect "network_name" password password_not_hexadecimal ifname wlxx hidden yes
$ nmcli d
DEVICE TYPE STATE CONNECTION
wlxx wifi connected network_name
...
$ sudo ifconfig
- local IP address after inet.


==== internal HDD with ext4
- connect SATA cable
- find device
$ sudo fdisk -l

- find UUID
$ sudo blkid /dev/sdb1

- add mount point
$ sudo mkdir /media/hdd0

- add mount in /etc/fstab
UUID=16d6fc51-ac58-478a-98b4-f5ee3d5fbccc /media/hdd0 ext4 defaults 0 2


==== usb external SSD with exFat
- add mount point
$ sudo mkdir /media/ssd0

$ sudo apt install exfat-fuse exfat-utils
- add mount in /etc/fstab, but noauto since it is optional
UUID=A6A9-5614 /media/ssd0 exfat noauto,uid=1000,gid=1000 0 2

- can mount after booting
$ sudo mount /dev/sdb1
OR
$ sudo mount -t exfat /dev/sdb1 /media/usb0 -o uid=1000


==== ~/bin/*
$ mkdir ~/bin
$ cp /media/hdd0/data/soh/bin/publish ~/bin/


================================================================================ nvidia

==== necessary packages needed for nvidia-driver installation
$ sudo apt install build-essential // for GCC
$ gcc -v
...
gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)

$ sudo apt install pkg-config
$ sudo apt install xorg
$ sudo apt install libglvnd-dev


==== NOTE: check compatibilities
https://www.tensorflow.org/install/source
https://www.tensorflow.org/install/gpu
https://mc.ai/tensorflow-gpu-installation-on-ubuntu-18-04/

Version Python version Compiler Build tools cuDNN CUDA
tensorflow-2.1.0 2.7, 3.5-3.7 GCC 7.3.1 Bazel 0.27.1 7.6 10.1

- gcc 7.3.1 is not 7.5.0 installed => try to ignore the difference
- bazel 2.0.0 is installed actually. Find the reason why in the following bazel section.


==== nvidia drivers
- the apt-intallable drivers are not recent:
$ sudo apt list |grep nvidia

- find compatible driver versions at https://www.nvidia.com/Download/Find.aspx?lang=en-us
440.64 released at Feb 28 is most recent
- download and install it
$ chmod +x NVIDIA-Linux-x86_64-440.64.run
$ sudo ./NVIDIA-Linux-x86_64-440.64.run


==== CUDA

- at https://developer.nvidia.com/cuda-toolkit-archive
click "CUDA Toolkit 10.1 update2" and select Linux,..., deb(network).
And then instructions appear:
$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
$ sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600
$ sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
$ sudo add-apt-repository "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/ /"
$ sudo apt-get update
$ sudo apt-get -y install cuda


==== cuDNN

- at https://developer.nvidia.com/rdp/cudnn-download
(nvidia membership login needed)
download 3 .deb files for Ubuntu 18.04
$ sudo dpkg -i ~/tmp/libcudnn7_7.6.5.32-1+cuda10.1_amd64.deb
$ sudo dpkg -i ~/tmp/libcudnn7-dev_7.6.5.32-1+cuda10.1_amd64.deb
$ sudo dpkg -i ~/tmp/libcudnn7-doc_7.6.5.32-1+cuda10.1_amd64.deb

$ vi ~/.bashrc
export CUDA_HOME="/usr/local/cuda"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${CUDA_HOME}/lib64:/usr/local/cuda/extras/CUPTI/lib64"
// cupti is installed with CUDA toolkit

$ vi ~/.profile
if [ -d "/usr/local/cuda/bin" ] ; then
PATH="/usr/local/cuda/bin:$PATH"
fi

$ source ~/.profile
// .bashrc is included in .profile currently


================================================================================ tensorflow

==== packages needed for tensorflow
// install pip3 in /usr/bin/
$ sudo apt install python3-dev python3-pip


/ install packages in ~/.local/lib/python3.6/site-packages/
// pip 20.0.2 is installed in here too.
// pip3 and pip are the same in here.
// This pip is used from now on.
$ pip3 install -U --user pip six numpy wheel setuptools mock 'future>=0.17.1'
$ source .profile
$ which pip
/home/soh/.local/bin/pip

$ pip install -U --user keras_applications --no-deps
$ pip install -U --user keras_preprocessing --no-deps


==== bazel
// I cloned tensorflow github and found bazel-2.0.0 is for the current tensorflow in tensorflow/configure.py
// as explained in https://www.tensorflow.org/install/source
// Note that it is different from 0.27.1 of the table in https://www.tensorflow.org/install/source
https://docs.bazel.build/versions/master/install-ubuntu.html#install-on-ubuntu

$ sudo apt install curl
$ curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -
$ echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
$ sudo apt update
$ sudo apt list |grep bazel
$ sudo apt install bazel-2.0.0
$ ln -s /usr/bin/bazel-2.0.0 ~/bin/bazel


==== tensorflow 2.x
https://www.tensorflow.org/install/source
$ git clone https://github.com/tensorflow/tensorflow.git
$ cd tensorflow
$ ./configure
// default values except CUDA
$ cd /usr/bin
$ sudo ln -s python3.6 python
// needed for bazel build
$ bazel build //tensorflow/tools/pip_package:build_pip_package
...
bazel-bin/tensorflow/tools/pip_package/build_pip_package
INFO: Elapsed time: 5277.459s, Critical Path: 205.99s
INFO: 22774 processes: 22774 local.
INFO: Build completed successfully, 35561 total actions

$ ./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
$ pip install --user /tmp/tensorflow_pkg/tensorflow-2.1.0-cp36-cp36m-linux_x86_64.whl
$ cd
// change directory to outside the TensorFlow source tree. This is necessary.
// check
$ python -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))"
...
tf.Tensor(71.211624, shape=(), dtype=float32)
$ python -c "import tensorflow as tf; print("tf.__version__")"
2.1.0
// NOTE: python is a link to python3


================================================================================ etc packages, modules

$ pip install -U --user librosa pandas bs4 matplotlib
$ sudo apt install python3-tk // necessary to show charts through ssh