CLSP Cluster Notes | Day 3

Installing fairseq

Fairseq is a Python-based language modeling toolkit devloped by Facebook (I was also recommended Marian, which is C++-based and developed by Microsoft, but fairseq was easier to install, and Python is all the rage these days).

For the record, the CLSP grid runs Python2 instead of Python3 as default, so I inevitably came across the problem at the last step:

git clone https://github.com/pytorch/fairseq
cd fairseq
pip install --editable ./

Just reminding myself to use pip3 instead of pip next time.

Fairseq command-line tools

Ran into a question about the difference between fairseq-generate & fairseq-interactive, and found this:

fairseq-generate is for batched data and fairseq-interactive is for streaming data (and interactive use). Otherwise they are very similar.

In this case, it would make more sense to use the former when writing scripts & scheduling jobs.