Commit 63b96b
2026-04-13 06:56:18 Anonymous: Initial commit| /dev/null .. indra startup checklist.md | |
| @@ 0,0 1,39 @@ | |
| + | # Indra startup checklist |
| + | |
| + | SSH into INDRA (192.168.40.40) as the user `isnai`. |
| + | If you SSH with another user, change users with `sudo -u isnai bash` (assuming you are among the sudoers) |
| + | |
| + | ## Sanity Checks |
| + | After booting check that all GPUs are working with `nvtop`. |
| + | |
| + | If they are not working or `nvtop` is not working, please reinstall nvtop, `nvidia-smi` and nvidia drivers in that order: try after each. |
| + | |
| + | You may have to restart again after the installation. |
| + | |
| + | ## Restart the LLM servers: |
| + | ```bash |
| + | cd /home/isnai/dev/tgi |
| + | nohup ./run-qwen32.sh > /home/isnai/qwen32.log 2>&1 & |
| + | nohup ./roleplay.sh > /home/isnai/roleplay.log 2>&1 & |
| + | |
| + | cd /home/isnai/dev/sglang/sglang |
| + | source env/bin/activate |
| + | nohup ./sglangrun.sh > /home/isnai/sglang.log 2>&1 & |
| + | ``` |
| + | |
| + | ## Deploy VISNA |
| + | ```bash |
| + | cd /home/isnai/testing/visna |
| + | source env/bin/activate |
| + | export CUDA_VISIBLE_DEVICES=6,7 |
| + | nohup ./gunicorn_backend.sh > backend.log 2>&1 & |
| + | nohup ./gunicorn_frontend.sh > frontend.log 2>&1 & |
| + | ``` |
| + | |
| + | ## Troubleshooting |
| + | You should be able to see [VISNA](https://visna.isn.edu.au/). |
| + | If notm here are other possible things you may have to do. |
| + | |
| + | 1. Restart nginx with `sudo systemctl restart nginx` |
| + | 2. Check if mongodb is running with `sudo systemctl status mongod` |
| + | 3. If is not running, run `sudo systemctl start mongod` |