CLSP Cluster Notes | Day 4

I struggled quite a bit with the fairseq commands today (and job scheduling). I realized that I could not run fairseq-interactive with qlogin since the latter was not granting me a session, and the former was too heavy a job. So, in the end I decided to non-interactively train a model instead.

fairseq-preprocess

I submitted a lot of jobs, but most of them failed to execute thoroughly. Most of it had to do with directories - specifying -wd proved to be an annoyance, and the place I wanted the output to be saved ended up being where some bash scripts were saved after execution as well, so the program had trouble looking for the necessary files it needed. In the end I gave up and set the working directory to fairseq/examples/translation and it worked out.

Next steps include actually training the model (and figuring out how to acquire GPUs) and then generating output.