A navigation simulator (navsim) API built on top of Python, Pytorch.

In the future, navsim may be compatible with a variety of simulators, but for now it uses A Realistc Open environment for Rapid Agent training(ARORA) Simulator, that is a highly attributed Unity3D GameEngine based Berlin city environment.

Getting started#

You can either run navsim inside container or directly in host without container.

Get the code#

clone the navsim repo:

git clone --recurse-submodules git@github.com:ucf-sttc/navsim.git

All further commands should be done inside navsim repo: cd navsim

Run navsim in the host (without container)#

Install pre-requisites for using in the host (without container)#

  • Install mamba 4.12.0-3 : https://github.com/conda-forge/miniforge/releases/download/4.12.0-3/Mambaforge-4.12.0-3-Linux-x86_64.sh

  • cd /path/to/navsim/repo
mamba create -n navsim python==3.8.16
mamba env update -n navsim -f pyenv/navsim.yml
conda activate navsim
./install-repo.sh

Run the experiments (in the host, without container)#

  • Read navsim_envs tutorial to use and test the navsim_envs

  • Run jupyter notebook. The notebooks are in examples folder.

  • Run the <navsim command> is described in the section below

The <navsim command> examples#

  • navsim --plan --env arora-v0 --show_visual --env_path /unity-envs/<path-to-arora-binary>. For example, <path-to-arora-binary> in our case is the foldername and binary after $HOME/unity-envs that we mapped in line 5 of docker-compose earlier: ARORA/ARORA.x86_64.

  • navsim_env_test min_env_config.yml

  • navsim --help shows the options

  • navsim --run_id $run_id --env_path $envdir/$envbin - executes and/or trains the model

  • navsim-benchmark $envdir/$envbin - benchmarks the model

  • navsim-saturate-gpu $envdir/$envbin - Saturates the GPU

  • Replace the navsim command with your own command if you are just importing the NavSim env and have your own code in experiment directory.

/opt/py/lib/python3.8/site-packages/torch/utils/tensorboard/__init__.py:4: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if not hasattr(tensorboard, "__version__") or LooseVersion(
usage: navsim [-h] [--run_id RUN_ID] [--env ENV] [--plan]
              [--rl_backend RL_BACKEND] [--debug] [--force]
              [--total_episodes TOTAL_EPISODES]
              [--train_interval TRAIN_INTERVAL]
              [--checkpoint_interval CHECKPOINT_INTERVAL]
              [--agent_gpu_id AGENT_GPU_ID]
              [--episode_max_steps EPISODE_MAX_STEPS] [--seed SEED]
              [--discount DISCOUNT] [--tau TAU]
              [--exploration_noise EXPLORATION_NOISE]
              [--batch_size BATCH_SIZE] [--memory_capacity MEMORY_CAPACITY]
              [--resume] [--continue] [--clear_memory]
              [--agent_car_physics AGENT_CAR_PHYSICS] [--area AREA]
              [--env_path ENV_PATH] [--env_gpu_id ENV_GPU_ID] [--goal GOAL]
              [--goal_distance GOAL_DISTANCE]
              [--goal_clearance GOAL_CLEARANCE] [--obs_mode OBS_MODE]
              [--obs_height OBS_HEIGHT] [--obs_width OBS_WIDTH]
              [--reward_for_goal REWARD_FOR_GOAL]
              [--reward_for_no_viable_path REWARD_FOR_NO_VIABLE_PATH]
              [--reward_step_mul REWARD_STEP_MUL]
              [--reward_collision_mul REWARD_COLLISION_MUL]
              [--reward_spl_delta_mul REWARD_SPL_DELTA_MUL] [--save_actions]
              [--save_visual_obs] [--save_vector_obs] [--show_visual]
              [--segmentation_mode SEGMENTATION_MODE] [--task TASK]
              [--terrain TERRAIN] [--timeout TIMEOUT]
              [--traffic_vehicles TRAFFIC_VEHICLES]
              [--mem_backend MEM_BACKEND]

optional arguments:
  -h, --help            show this help message and exit

Run Configuration:
  The arguments are used to configure the runtime execution.

  --run_id RUN_ID       The identifier for the training run. This identifier
                        is used to name the subdirectories in which the
                        trained model and summary statistics are saved as well
                        as the saved model itself. If you use TensorBoard to
                        view the training statistics, always set a unique run-
                        id for each training run. (The statistics for all runs
                        with the same id are combined as if they were produced
                        by a the same session.) (default: demo)
  --env ENV             EnvironmentL arora-v0 or ride-v0 (default: arora-v0)
  --plan                Run with Planner (default: False)
  --rl_backend RL_BACKEND
                        The backend library for RL. (default: navsim)
  --debug               Turn debugging on (default: False)
  --force               Whether to force-overwrite this run-id's existing
                        summary and model data. (Without this flag, attempting
                        to train a model with a run-id that has been used
                        before will throw an error. (default: False)
  --total_episodes TOTAL_EPISODES
                        Total number of episodes to run. If resume is used,
                        then it will try to read the previously run episodes
                        and continue from there. (default: 1)
  --train_interval TRAIN_INTERVAL
                        Train the model after these many global steps. If set
                        to 0 then model wont train (default: 1)
  --checkpoint_interval CHECKPOINT_INTERVAL
                        Execute the episodes in blocks of checkpoint intervals
                        (default: 1)
  --agent_gpu_id AGENT_GPU_ID
                        Which GPU to run models in agent on (default: 0)
  --episode_max_steps EPISODE_MAX_STEPS
                        Maximum number of steps in an Episode, aka Episode
                        Length (default: 50)
  --seed SEED           Seed (default: None)
  --discount DISCOUNT   discount (default: 0.99)
  --tau TAU             tau (default: 0.005)
  --exploration_noise EXPLORATION_NOISE
                        exploration_noise (default: 0.1)
  --batch_size BATCH_SIZE
                        Batch Size (default: 32)
  --memory_capacity MEMORY_CAPACITY
                        Total capacity of memory, should be > batch_size
                        (default: 100)
  --resume              resumes the interrupted training from last saved
                        checkpoint (default: False)
  --continue            continue the training forward from last saved
                        checkpoint (default: False)
  --clear_memory        On resuming, the memory will be cleared and actions
                        would be sampled from the neural network, if neural
                        network is not trained yet, then actions would sample
                        from action_space (default: False)

Environment Configuration:
  The arguments are used to configure the environment.

  --agent_car_physics AGENT_CAR_PHYSICS
                        Agent Car Physics Levels : 0,1,2,10 (default: 0)
  --area AREA           0 for all map, 1 for train area (0, 0 to 500, 3000), 2
                        for test area (500, 0 to 4000, 3000) (default: 0)
  --env_path ENV_PATH   Path to the Unity executable to train (default: None)
  --env_gpu_id ENV_GPU_ID
                        Which GPU to run env on (default: 0)
  --goal GOAL           Goal (default: 0)
  --goal_distance GOAL_DISTANCE
                        Distance to goal from current location (default: 50)
  --goal_clearance GOAL_CLEARANCE
                        Sets the minimum clear area around a point for it to
                        be a valid location to place the goal (default: 2.5)
  --obs_mode OBS_MODE   Observation Mode : 0,1,2 (default: 0)
  --obs_height OBS_HEIGHT
                        Observation height (default: 64)
  --obs_width OBS_WIDTH
                        Observation width (default: 64)
  --reward_for_goal REWARD_FOR_GOAL
                        Reward for reaching the goal (default: 50)
  --reward_for_no_viable_path REWARD_FOR_NO_VIABLE_PATH
                        Reward for no more viable paths remaining to reach the
                        goal (default: -50)
  --reward_step_mul REWARD_STEP_MUL
                        Multiply step reward [which is -(goal_reward /
                        spl_start) with this number (default: 0.1)
  --reward_collision_mul REWARD_COLLISION_MUL
                        Multiply step reward with this number when there is a
                        collision and add to reward (default: 4)
  --reward_spl_delta_mul REWARD_SPL_DELTA_MUL
                        Shortest path length delta multiplier (default: 1)
  --save_actions        Save the actions vector at every step (default: True)
  --save_visual_obs     Save the visual observations at every step (default:
                        True)
  --save_vector_obs     Save the vector observations at every step (default:
                        True)
  --show_visual         Show visual obs window (default: False)
  --segmentation_mode SEGMENTATION_MODE
                        Segmentation Mode (default: 1)
  --task TASK           Task (default: 0)
  --terrain TERRAIN     Terrain: could be 0 or 1 (default: 0)
  --timeout TIMEOUT     TimeOut for the Env (default: 600)
  --traffic_vehicles TRAFFIC_VEHICLES
                        Number of traffic vehicles (default: 0)

Developer Configuration:
  The arguments are used by developers to benchmark and debug navsim API

  --mem_backend MEM_BACKEND
                        The backend library for Rollback Memory. Options cupy,
                        numpy. (default: cupy)

Errors FAQ#

vulkan error while starting the container#

ERROR: [Loader Message] Code 0 : /usr/lib/x86_64-linux-gnu/libvulkan_radeon.so: cannot open shared object file: No such file or directory
No protocol specified
No protocol specified
ERROR: [Loader Message] Code 0 : loader_scanned_icd_add: Could not get 'vkCreateInstance' via 'vk_icdGetInstanceProcAddr' for ICD libGLX_nvidia.so.0
ERROR: [Loader Message] Code 0 : /usr/lib/i386-linux-gnu/libvulkan_intel.so: cannot open shared object file: No such file or directory
ERROR: [Loader Message] Code 0 : /usr/lib/i386-linux-gnu/libvulkan_radeon.so: cannot open shared object file: No such file or directory
ERROR: [Loader Message] Code 0 : /usr/lib/x86_64-linux-gnu/libvulkan_intel.so: cannot open shared object file: No such file or directory
ERROR: [Loader Message] Code 0 : /usr/lib/i386-linux-gnu/libvulkan_lvp.so: cannot open shared object file: No such file or directory
ERROR: [Loader Message] Code 0 : /usr/lib/x86_64-linux-gnu/libvulkan_lvp.so: cannot open shared object file: No such file or directory
Cannot create Vulkan instance.
This problem is often caused by a faulty installation of the Vulkan driver or attempting to use a GPU that does not support Vulkan.
ERROR at /build/vulkan-tools-oFB8Ns/vulkan-tools-1.2.162.0+dfsg1/vulkaninfo/vulkaninfo.h:666:vkCreateInstance failed with ERROR_INCOMPATIBLE_DRIVER

Solution: For fixing this error you have to update your nvidia driver and fix the id inside the container, as follows:

  1. Check your nvidia driver with the following commands: sudo apt list --installed | grep nvidia-driver and nvidia-smi

For example on our laptop:

armando@thunderbird:~/workspace/navsim$ sudo apt list --installed | grep nvidia-driver

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

nvidia-driver-470/focal-updates,focal-security,now 470.182.03-0ubuntu0.20.04.1 amd64 [installed]

armando@thunderbird:~/workspace/navsim$ nvidia-smi
Sun May 14 10:53:30 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.182.03   Driver Version: 470.182.03   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+

Reinstall the nvidia-driver or update it to latest one.

sudo apt update
sudo apt install nvidia-driver-530
sudo reboot

If you dont rebooot after installing the driver then you will get the following error:

Failed to initialize NVML: Driver/library version mismatch

  1. Update the id inside the container as per section: fix the user id inside the container

Contributing to NavSim API#

build and test flow#

  • Modify code

release flow#

  • switch to feature branch

  • Modify the version.txt

  • Modify image version in:

    • docker-compose.yml

    • .github/workflows/deploy-docs.yml

  • Build the container: docker compose build navsim-build

  • Run fixid: DUID=``id -u`` DGID=``id -g`` docker compose build navsim-fixid

  • Test the container

  • Commit and push the changes

  • create a pull request to main branch

  • Merge the pull request

  • Switch to main branch: git checkout main

  • Build the container: docker compose build navsim-build

  • Push the container: docker compose push navsim-build

  • git tag vx.x.x and git push --tags

  • Run the docs workflow in github manually

  • Create a release in github with the tag

General dev info:#

  • Use only google style to document your code: https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html#example-google

  • /opt/navsim : Navsim code folder is kept here

to modify and build the docs:#

  1. Create <version>-docs branch from master or version branch

  2. Make sure you have navsim conda env activated and local repo installed with pip.

  3. Modify the .md files in /path/to/navsim, /path/to/navsim-lab, /path/to/navsim-envs

  4. go to navsim/docs

  5. run make html latexpdf. If still errors, please call me and I will help fix

  6. pdf and html versions are inside the docs/build/... folders

  7. If you are happy with the PDF/formatting etc then commit and push the doc branch back

misc tasks:#

To give the zip of repo to someone, run the following command:

zip -FSr repo.zip \
    navsim-lab navsim-envs \
    navsim-mlagents/ml-agents-envs \
    navsim-mlagents/gym-unity \
    version.txt \
    install-repo.sh \
    examples \
    -x@.dockerignore