The camera assembly is defined on the following website page.

Register the server’s public key:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE || sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE

Add the server to the list of repositories:

sudo add-apt-repository "deb https://librealsense.intel.com/Debian/apt-repo $(lsb_release -cs) main" -u

Install the libraries

sudo apt-get install librealsense2-dkms
sudo apt-get install librealsense2-utils

Install the following libraries (optionally):

sudo apt-get install librealsense2-dev
sudo apt-get install librealsense2-dbg

To finish:

sudo apt-get update
sudo apt-get upgrade

You can reconnect your camera and run:

realsense-viewer

This example demonstrates how to start streaming depth frames from the camera and display the image in the console as an ASCII art.

This link provided the example. I modified slightly the script :

## License: Apache 2.0. See LICENSE file in root directory.
## Copyright(c) 2015-2017 Intel Corporation. All Rights Reserved.

#####################################################
## librealsense tutorial #1 - Accessing depth data ##
#####################################################

import time
# First import the library
import pyrealsense2 as rs

try:
    # Create a context object. This object owns the handles to all connected realsense devices
    pipeline = rs.pipeline()

    # Configure streams
    config = rs.config()
    config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)

    # Start streaming
    pipeline.start(config)

    while True:
        # This call waits until a new coherent set of frames is available on a device
        # Calls to get_frame_data(...) and get_frame_timestamp(...) on a device will return stable values until wait_for_frames(...) is called
        frames = pipeline.wait_for_frames()
        depth = frames.get_depth_frame()
        if not depth: continue

        # Print a simple text-based representation of the image, by breaking it into 10x20 pixel regions and approximating the coverage of pixels within one meter
        coverage = [0]*64
        for y in range(480):
            for x in range(640):
                dist = depth.get_distance(x, y)
                if 0 < dist and dist < 1:
                    coverage[x//10] += 1
            
            if y%20 is 19:
                line = ""
                for c in coverage:
                    line += " .:nhBXWW"[c//25]
                coverage = [0]*64
                print(line)
            time.sleep(1)
    exit(0)
#except rs.error as e:
#    # Method calls agaisnt librealsense objects may throw exceptions of type pylibrs.error
#    print("pylibrs.error was thrown when calling %s(%s):\n", % (e.get_failed_function(), e.get_failed_args()))
#    print("    %s\n", e.what())
#    exit(1)
except Exception as e:
    print(e)
    pass

The result is rather fun :

This example display the same videos as in the RealSense viewer (with the command realsense).

The code is available here.

This is the following result :

This example remove the background thanks to the depth.

• Download and install Raspberry Pi Imager to a computer with an SD card reader.
• Choose the Ubuntu Server 20.04.4 LTS (RPI 3/4/400) 32 bits. Indeed, Noetic is available only on Ubuntu 20.04.4.
• Then, choose the SD card and write on it.
• Click on the setting button
• Set hostname raspberrypi.local
• Click on Enable SSH and Configure Wireless LAN

Open a terminal and run:

ssh pi@raspberrypi.local

Solution

The terminal displays :

The authenticity of host 'raspberrypi.local (192.168.0.119)' can't be established.
ECDSA key fingerprint is SHA256:13zUWPqvlgOGa3yJ/oHvZIC8bnDb0gw7p86L9PNVEp4.
Are you sure you want to continue connecting (yes/no/[fingerprint])? }

Answer yes.

pi@raspberrypi.local's password:

Answer the previous password.

Setup your sources.list

sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'

Set up your keys

sudo apt install curl # if you haven't already installed curl
curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -

Then run:

sudo apt update

Install ROS packages without GUI tools :

sudo apt install ros-noetic-ros-base

Don’t forget to add the following command in your .bashrc :

source /opt/ros/noetic/setup.bash

Install all dependencies

sudo apt-get install -y python-rosdep python-rosinstall-generator python-wstool python-rosinstall build-essential cmake

Swap is a storage on a hard disk where the operating system store temporally data. It is required when there aren’t enough spaces in the RAM.

The following command checks if there is a swap available :

sudo swapon --show

Run (to check if a swap is running):

free -h

Create a swapfile of 1Go :

sudo fallocate -l 1G /swapfile

You can check is the command is a success with :

ls -lh /swapfile

The file is available only for the root :

sudo chmod u=rw /swapfile

The space is defined like a swap space :

sudo mkswap /swapfile

Enable the swap :

sudo swapon /swapfile

You can check if the swap is working with:

free -h

These following commands allows the swap to work each time :

sudo cp /etc/fstab /etc/fstab.bak
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

mkdir -p ~/catkin_ws/src
cd ~/catkin_ws/
catkin_make

If you have the following error : No CMAKE_CXX_COMPILER could be found, run:

sudo apt-get update && sudo apt-get install build-essential
sudo apt-get install python3-catkin-tools

Clone the following packages in the directory src:

git clone https://github.com/robotont/robotont_description
git clone https://github.com/robotont/robotont_nuc_description
git clone https://github.com/robotont/robotont_navigation
git clone https://github.com/robotont/robotont_demos
git clone https://github.com/robotont/robotont_msgs
git clone https://github.com/ros-teleop/teleop_twist_keyboard
git clone https://github.com/robotont/robotont_laserscan_to_distance
sudo apt-get install ros-noetic-joy

Install a package for the drivers in your source :

git clone https://github.com/robotont/robotont_driver
cd robotont_driver
rosinstall . .ci-rosinstall 
cd serial 
make 
make install
cd ..
cd ..
mv robotont_driver/serial .

Then, run from your workspace:

catkin_make

The following packages are needed to run the demos :

1. For mapping 2D:

sudo apt update
sudo apt install ros-noetic-depthimage-to-laserscan
sudo apt install ros-noetic-cartographer-ros
sudo apt install ros-noetic-move-base

2. For 3D mapping :

sudo apt install ros-noetic-rtabmap-ros

3. For AR tracking, in the directory src

git clone -b noetic-devel https://github.com/machinekoder/ar_track_alvar/tree/noetic-devel‌

Follow the same steps of the Ubuntu 20.04.4 installation part. However, choose the Raspberry Pi OS Lite (A port of Debian Buster with security updates and no desktop environment)

Setup your sources.list

sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'

Set up your keys

sudo apt install curl # if you haven't already installed curl
curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -

Then run:

sudo apt update

If you have an error, run these commands:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F42ED6FBAB17C654
sudo apt update

Install all dependencies

sudo apt-get install -y python-rosdep python-rosinstall-generator python-wstool python-rosinstall build-essential cmake

Initialize Rosdep

sudo rosdep init

Then, update

 rosdep update

Run

mkdir ~/catkin_ws
cd ~/catkin_ws

Use rosinstall_generator to generate a list of noetic dependencies :

rosinstall_generator ros_comm --rosdistro noetic --deps --wet-only --tar > noetic-ros_comm-wet.rosinstall

Run the following command:

wstool init src noetic-ros_comm-wet.rosinstall

Install all system dependencies

rosdep install -y --from-paths src --ignore-src --rosdistro noetic -r --os=debian:buster

Follow instructions from swap part, then write the following command :

sudo src/catkin/bin/catkin_make_isolated --install -DCMAKE_BUILD_TYPE=Release --install-space /opt/ros/noetic -j1 -DPYTHON_EXECUTABLE=/usr/bin/python3

Install the Robotont’s package

git clone https://github.com/robotont/robotont_description
git clone https://github.com/robotont/robotont_nuc_description
git clone https://github.com/robotont/robotont_navigation
git clone https://github.com/robotont/robotont_demos
git clone https://github.com/robotont/robotont_driver

Run

sudo apt-get install libspnav-dev
sudo apt install libx11-dev

From this repository, download in your your directory src the following directories :

sensor_msg
geometry_msg

Git clone the following repositories in your src:

git clone https://github.com/ros-drivers/joystick_drivers
git clone https://github.com/ros/bond_core
git clone https://github.com/ros/roslint

Add the following lines in your .bashrc :

source /opt/ros/noetic/setup.bash
source ~/catkin_ws/devel/setup.bash

From this repository, download the following directories in your src:

nav_msgs
actionlib_msgs

In the same way, from this repository :

tf

Install the package angle in your src :

git clone https://github.com/ros/angles

From this repository, download the following directories in your src :

tf2
tf2_msgs
tf2_ros
tf2_py

Install the packages actionlib and serial in your src :

git clone https://github.com/ros/actionlib
git clone https://github.com/wjwwood/serial

In the directory serial, run the following commands to intall :

make
make test
make install

Then, run from your workspace:

catkin_make

If your have the following error :

-- tf: 1 messages, 1 services
CMake Error at catkin/cmake/test/gtest.cmake:180 (add_executable):
  add_executable cannot create target "test_transform_datatypes" because
  another target with the same name already exists.  The existing target is
  an executable created in source directory
  "/home/flynn/dev/ros/noetic/src/geometry2/tf2".  See documentation for
  policy CMP0002 for more details.
Call Stack (most recent call first):
  catkin/cmake/test/gtest.cmake:89 (_catkin_add_executable_with_google_test)
  catkin/cmake/test/gtest.cmake:37 (_catkin_add_google_test)
  geometry/tf/CMakeLists.txt:86 (catkin_add_gtest)


CMake Error at geometry/tf/CMakeLists.txt:87 (target_link_libraries):
  Attempt to add link library "tf" to target "test_transform_datatypes" which
  is not built in this directory.

  This is allowed only when policy CMP0079 is set to NEW.

You need to change the file CMakeLists.txt from src/geometry2/tf2as here.

Your CMakefile looks like :

cmake_minimum_required(VERSION 3.0.2)
project(tf2)

find_package(console_bridge REQUIRED)
find_package(catkin REQUIRED COMPONENTS geometry_msgs rostime tf2_msgs)
find_package(Boost REQUIRED COMPONENTS system thread)

catkin_package(
   INCLUDE_DIRS include
   LIBRARIES tf2
   DEPENDS console_bridge
   CATKIN_DEPENDS geometry_msgs tf2_msgs rostime)

include_directories(include ${catkin_INCLUDE_DIRS} ${console_bridge_INCLUDE_DIRS})

# export user definitions

#CPP Libraries
add_library(tf2 src/cache.cpp src/buffer_core.cpp src/static_cache.cpp)
target_link_libraries(tf2 ${Boost_LIBRARIES} ${catkin_LIBRARIES} ${console_bridge_LIBRARIES})
add_dependencies(tf2 ${catkin_EXPORTED_TARGETS})

install(TARGETS tf2
  ARCHIVE DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
  LIBRARY DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
  RUNTIME DESTINATION ${CATKIN_GLOBAL_BIN_DESTINATION}
)

install(DIRECTORY include/${PROJECT_NAME}/
  DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION}
)

# Tests
if(CATKIN_ENABLE_TESTING)

catkin_add_gtest(test_cache_unittest test/cache_unittest.cpp)
target_link_libraries(test_cache_unittest tf2  ${console_bridge_LIBRARIES})
add_dependencies(test_cache_unittest ${catkin_EXPORTED_TARGETS})

catkin_add_gtest(test_static_cache_unittest test/static_cache_test.cpp)
target_link_libraries(test_static_cache_unittest tf2  ${console_bridge_LIBRARIES})
add_dependencies(test_static_cache_unittest ${catkin_EXPORTED_TARGETS})

catkin_add_gtest(test_simple test/simple_tf2_core.cpp)
target_link_libraries(test_simple tf2  ${console_bridge_LIBRARIES})
add_dependencies(test_simple ${catkin_EXPORTED_TARGETS})

add_executable(speed_test EXCLUDE_FROM_ALL test/speed_test.cpp)
target_link_libraries(speed_test tf2  ${console_bridge_LIBRARIES})
add_dependencies(tests speed_test)
add_dependencies(tests ${catkin_EXPORTED_TARGETS})

catkin_add_gtest(test_transform_datatypes2 test/test_transform_datatypes.cpp)
target_link_libraries(test_transform_datatypes2 tf2  ${console_bridge_LIBRARIES})
add_dependencies(test_transform_datatypes2 ${catkin_EXPORTED_TARGETS})

endif()

Run again

catkin_make

The project required laser_scan_to_distance. Thus, run in your directory src:

git clone https://github.com/ros-perception/depthimage_to_laserscan
git clone https://github.com/ros/dynamic_reconfigure

From this repository, dowload in your src:

image_geometry

In the same way, from this repository:

image_transport

From this repository:

nodelet

The project required also some packages to do slam. These packages allow the robot to do simultaneous localization and mapping.

Install in your src:

git clone https://github.com/ros-perception/slam_gmapping
git clone https://github.com/ros-perception/openslam_gmapping

The robot needs the package move_base. It is useful for the navigation.

From this repository, dowload in your src:

Install in your src:

git clone https://github.com/ros-perception/laser_geometry
https://github.com/ros-planning/navigation_msgs

From this repository, dowload the following directories in your src:

move_base
base_local_planner
costmap_2d
voxel_grid
nav_core
clear_costmap_recovery
nvfn
rotate_recovery

In the same way, from this repository:

tf2_geometry_msgs
tf2_sensor_msgs

From this repository:

viualization_msgs

Install the following package to control the robot with your keyboard :

git clone https://github.com/ros-teleop/teleop_twist_keyboard

Run the last command :

catkin_make

Add packages to your system:

echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list

curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

sudo apt-get update

Install the Edge TPU runtime:

sudo apt-get install libedgetpu1-std

Connect the USB Accelerator to your computer using the provided USB 3.0 cable.

Install the PyCoral library

sudo apt-get install python3-pycoral

Download the following code :

mkdir coral && cd coral

git clone https://github.com/google-coral/pycoral.git

cd pycoral

Download the following resources :

bash examples/install_requirements.sh classify_image.py

Run the image classifier with the bird photo

python3 examples/classify_image.py \
--model test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \
--labels test_data/inat_bird_labels.txt \
--input test_data/parrot.jpg

The result must be :

----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
11.8ms
3.0ms
2.8ms
2.9ms
2.9ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.75781

Branch the camera.

Enable the camera

sudo raspi-config

Click on Interface Option

Then, click on P1 Camera :

Click on yes

Run the following command :

vcgencmd get_camera

If you have :

supported=1 detected=1

It works !

Install Tensorflow lite interpreter

sudo apt-get install python3-tflite-runtime

Download a sample project

mkdir google-coral && cd google-coral
git clone https://github.com/google-coral/examples-camera --depth 1

Load the pre-trained model :

cd examples-camera
sh download_models.sh

Install requirements

cd opencv
bash install_requirements.sh

Modify detect.py as the following file:

# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""A demo that runs object detection on camera frames using OpenCV.

TEST_DATA=../all_models

Run face detection model:
python3 detect.py \
  --model ${TEST_DATA}/mobilenet_ssd_v2_face_quant_postprocess_edgetpu.tflite

Run coco model:
python3 detect.py \
  --model ${TEST_DATA}/mobilenet_ssd_v2_coco_quant_postprocess_edgetpu.tflite \
  --labels ${TEST_DATA}/coco_labels.txt

"""
import argparse
import cv2
import os

from pycoral.adapters.common import input_size
from pycoral.adapters.detect import get_objects
from pycoral.utils.dataset import read_label_file
from pycoral.utils.edgetpu import make_interpreter
from pycoral.utils.edgetpu import run_inference

def main():
    default_model_dir = '../all_models'
    default_model = 'mobilenet_ssd_v2_coco_quant_postprocess_edgetpu.tflite'
    default_labels = 'coco_labels.txt'
    parser = argparse.ArgumentParser()
    parser.add_argument('--model', help='.tflite model path',
                        default=os.path.join(default_model_dir,default_model))
    parser.add_argument('--labels', help='label file path',
                        default=os.path.join(default_model_dir, default_labels))
    parser.add_argument('--top_k', type=int, default=3,
                        help='number of categories with highest score to display')
    parser.add_argument('--camera_idx', type=int, help='Index of which video source to use. ', default = 0)
    parser.add_argument('--threshold', type=float, default=0.1,
                        help='classifier score threshold')
    args = parser.parse_args()

    print('Loading {} with {} labels.'.format(args.model, args.labels))
    interpreter = make_interpreter(args.model)
    interpreter.allocate_tensors()
    labels = read_label_file(args.labels)
    inference_size = input_size(interpreter)

    cap = cv2.VideoCapture(args.camera_idx)

    while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            break
        cv2_im = frame
        cv2_im_rgb = cv2.cvtColor(cv2_im, cv2.COLOR_BGR2RGB)
        cv2_im_rgb = cv2.resize(cv2_im_rgb, inference_size)
        run_inference(interpreter, cv2_im_rgb.tobytes())

        objs = get_objects(interpreter, args.threshold)[:args.top_k]
        for obj in objs:
                if is_detected_object(obj):
                        print(f"Object detected : {labels.get(obj.id, obj.id)}, confidence rate : {get_score_object(obj)} %")
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

    cap.release()
    cv2.destroyAllWindows()

def is_detected_object(obj):
        return get_score_object(obj) > 70

def get_score_object(obj):
        return int(100*obj.score)

def get_name_object(obj):
        return labels.get(obj.id, obj.id)

if __name__ == '__main__':
    main()

Run

python3 detect.py

You can lead your camera to an object as a bottle and you should get:

object detected : bottle, confidence rate : 83 %

Start with the following commands :

sudo apt-get update && sudo apt-get dist-upgrade
sudo apt-get install automake libtool vim cmake libusb-1.0-0-dev libx11-dev xorg-dev libglu1-mesa-dev

Create a new udev rule:

cd ~
git clone https://github.com/IntelRealSense/librealsense.git
cd librealsense
sudo cp config/99-realsense-libusb.rules /etc/udev/rules.d/ 

Apply the changes:

sudo su
udevadm control --reload-rules && udevadm trigger
exit

Add to the .bashrc the following line:

export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

Apply the changes:

source ~/.bashrc

Install protobuf:

cd ~
git clone --depth=1 -b v3.10.0 https://github.com/google/protobuf.git
cd protobuf
./autogen.sh
./configure
make -j1
sudo make install
cd python
export LD_LIBRARY_PATH=../src/.libs
python3 setup.py build --cpp_implementation 
python3 setup.py test --cpp_implementation
sudo python3 setup.py install --cpp_implementation
export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp
export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION_VERSION=3
sudo ldconfig
protoc --version

Install libtbb-dev:

cd ~
wget https://github.com/PINTO0309/TBBonARMv7/raw/master/libtbb-dev_2018U2_armhf.deb
sudo dpkg -i ~/libtbb-dev_2018U2_armhf.deb
sudo ldconfig
rm libtbb-dev_2018U2_armhf.deb

Install RealSense SDK librealsense :

cd ~/librealsense
mkdir  build  && cd build
cmake .. -DBUILD_EXAMPLES=true -DCMAKE_BUILD_TYPE=Release -DFORCE_LIBUVC=true
make -j1
sudo make install

Install RealSense SDK pyrealsense2 Pytho bindings for librealsense :

cd ~/librealsense/build
cmake .. -DBUILD_PYTHON_BINDINGS=bool:true -DPYTHON_EXECUTABLE=$(which python3)
make -j1
sudo make install

Add the following line to your .bashrc:

export PYTHONPATH=$PYTHONPATH:/usr/local/lib

Apply the change:

source ~/.bahsrc

Install OpenGL :

sudo apt-get install python-opengl
sudo -H pip3 install pyopengl
sudo -H pip3 install pyopengl_accelerate==3.1.3rc1

Run

sudo raspi-config

Choose the following option

Advanced Options
A2 Gl Driver
G2 GL (Fake KMS)

Dowload create in the directory debug in your raspberry pi the following python script:

## License: Apache 2.0. See LICENSE file in root directory.
## Copyright(c) 2015-2017 Intel Corporation. All Rights Reserved.

#####################################################
## librealsense tutorial #1 - Accessing depth data ##
#####################################################

# First import the library
import pyrealsense2.pyrealsense2 as rs

try:
    # Create a context object. This object owns the handles to all connected realsense devices
    pipeline = rs.pipeline()

    # Configure streams
    config = rs.config()
    config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)

    # Start streaming
    pipeline.start(config)

    while True:
        # This call waits until a new coherent set of frames is available on a device
        # Calls to get_frame_data(...) and get_frame_timestamp(...) on a device will return stable values until wait_for_frames(...) is called
        frames = pipeline.wait_for_frames()
        depth = frames.get_depth_frame()
        if not depth: continue

        # Print a simple text-based representation of the image, by breaking it into 10x20 pixel regions and approximating the coverage of pixels within one meter
        coverage = [0]*64
        for y in range(480):
            for x in range(640):
                dist = depth.get_distance(x, y)
                if 0 < dist and dist < 1:
                    coverage[x//10] += 1
            
            if y%20 is 19:
                line = ""
                for c in coverage:
                    line += " .:nhBXWW"[c//25]
                coverage = [0]*64
                print(line)
    exit(0)
#except rs.error as e:
#    # Method calls agaisnt librealsense objects may throw exceptions of type pylibrs.error
#    print("pylibrs.error was thrown when calling %s(%s):\n", % (e.get_failed_function(), e.get_failed_args()))
#    print("    %s\n", e.what())
#    exit(1)
except Exception as e:
    print(e)
    pass

Then run it, you should get this photo: