UnCloud docs

Introduction Watch Tech Open Source

Introduction

UnCloud is peer-to-peer chat and file transfer for Windows, MacOS, Android, iOS, Linux, Chromebook, and almost any platform with a modern browser.
UnCloud Watch lets you use a smartphone, laptop, or Chromebook as a remote security camera.
UnCloud and UnCloud Watch use encrypted WebRTC for direct communication between browsers.
Your chats, files, and video are not uploaded to the cloud or stored on a server.
The chat history exists only in the browsers, and goes away when the participants close the page.
Each chat session has a host and one or more guests. The host controls who can join and remain in the chat.
The host can offer files to guests, and guests can offer files to the host. Multi-gigabyte files can be sent.

How to host an UnCloud chat

Click to open UnCloud:
From the main page, click the "Share files or host chat" button.
If you want to share files, click the Browse or Select button on the "share files" line and choose the files.
If you want to require a password, enter it in the password box.
Click the "Start sharing" button. A join URL and access code will be shown.
Send the URL or access code to your guests by text, email, or any other method. Send the password if you set one.
Messages will appear in the large white box when guests join.

How to join an UnCloud chat

If you received a join URL, click on it and you will be joined automatically.
If you received an access code, go to the UnCloud main page, click the "Get Files or Join Chat" button, and enter the code to join.
Enter the password if one is required.

File sharing

Both the host and guest users have a Browse or Choose Files or Select button.
Files offered by the host will display under the guests' chat boxes. Files offered by the guest will display under the host's chat box.
Click the Get button to download a file. On some browsers, you may be immediately prompted for a file save location.
On other browsers, the Get button turns into a Save button after the download. Clicking Save promtps for a location.
You can start more downloads while one is under way. If they show as Queued they will be done one at a time.
Guest cannot send each other files. The host can save and re-share a file from one guest so all others can download it.
Since this system is peer-to-peer, there is no cloud provider to virus scan the files. Be careful what you open!

UnCloud Watch lets you use a smartphone, laptop, or Chromebook as a remote security camera.
UnCloud Watch use encrypted WebRTC for direct communication between browsers.
No software installation is required on either the camera or viewer device.
Chrome and Safari browsers work well. The camera can stop and start on its own once permission has been granted.
Firefox is a problem, because it requires the window to be foreground to start the camera.
Although not designed as a video phone, if you set up a session in each direction, you can have a two-way conversation.

How to set up a remote camera

Attach the device to external power.
Adjust your power management settings so your device does not sleep. See below under "Power Management" for details.
Click to open UnCloud Watch:
Click the "Stream Device Camera" button.
Allow permission to use the camera. A video window will appear. Adjust the camera position.
If you have multiple cameras, an option will appear to choose a camera.
If you want to require a password, enter it in the password box. Without a password, anyone can view your camera.
If you want to assign a name to your feed, enter the name in the "Feed name" box.
If you want to transmit audio, check the "Permit Audio" button. In many places, video security is legal but audio is not.
Click "Start Streaming"
Save the access code or watch link to access your stream remotely.
See below "Power Management" for information about the "Show video", "Monitor battery", and "Stay awake" options.

How to view a remote camera

If you received a join URL, click on it and you will be joined automatically.

You may have to click the play arrow in the video window when it appears. Some browsers do not permit auto-play video.

If you have an access code, go to UnCloud Watch, click the "Watch remote video" button, and enter the code.
If a password is required, you will be prompted.
The video window will appear. If the video does not start, click the play arrow in the video window.
If the server is transmitting audio, you have to unmute the audio in the video window. Browsers do not allow auto-play audio.
You can change the resolution, choose the camera if there is more than one, or choose the audio source if there is more than one.
The host can only provide one feed. If you change settings, any other viewers will also be affected.

Power Management

If an UnCloud Watch feed is down, the main cause is that the machine went to sleep.
There are platform-specific settings to keep a machine from sleeping.
You will always need external power to maintain a video feed.

Power Management - ChromeOS

Start button/Settings/Device/Power
Under When idle/While charging, set to "Turn off display"

Power Management - Windows 10

Start Menu/Gear icon/Settings/System/Power & Sleep
Under "Sleep" heading, find "When plugged in, PC goes to sleep after" and set to "Never"
The Windows 7 Control Panel settings below also work.

Power Management - Windows 7

Start Menu/Control Panel/Power Options
Change Plan Settings
Under "Plugged in" set "Put the computer to sleep" to "Never".

Power Management - MacOS

Apple menu/System Preferences/Battery/Power Adapter
Check box: "Prevent computer from sleeping automatically when the display is off"

Power Management - Android

Enable developer options - repeatedly click the build number in Settings.
Settings/Developer options/Stay awake (Screen will never sleep while charging)
Android can also be kept awake using the "Show video" option on the Host page.

The "Show video" option uses more power and may reduce frame rate.
Use this if the developer option does not keep the phone awake.

Set the phone to stay awake in developer mode. Turn down screen brightness to save power.
Increase screen timeout in phone display settings to at least two minutes, preferably five.
Set the "Stay Awake" option in UnCloud Host mode.

While the phone is on external power, developer stay awake is more efficient than "Show video" in UnCloud.
When external power is lost, "Stay Awake" mode automatically turns on "Show video" option.
This keeps the phone awake on internal battery, providing more observation time.

Disable automatic phone updates if possible. An update will reboot the phone and disable your camera feed.

Device choice

If AC power and WiFi are available, Chromebooks are a good choice.

Reliable Linux kernel, solid state, no moving parts.
Obsolete ones are available at low prices.
An external webcam can be plugged into a Chromebook to provide a second point of view.

If you do not have AC power and WiFi, Android phones are the best choice.

Prepaid mobile is an option where WiFi is not.
Android phones can be powered from batteries easily. See below.
You can remotely switch between the front and rear cameras.

Mac and PC laptops work, Firefox is problematic, use Chrome or Safari
iOS devices not thoroughly tested, I don't have any of those.
- Results from people testing on iOS: basic video streaming works.
- You may need to click on "Live Broadcast" to start viewing.
- Pinch down to see entire window.

Android with external battery power

Where AC power is not available, Android phones can easily be powered by batteries.
Commercial power bank

Power bank has a lithium battery and a regulator to provide USB power. Larger ones can generally last for a day or so.
Phones and power banks are sold <50% charged to extend shelf life of batteries. If you are buying new equipment, allow time to fully charge everything.
Power bank may turn off under low load. Make sure your power bank stays on after the phone battery finishes charging.

Car charger and 12-volt battery

Large lead-acid battery can operate a phone for days.
If your observation point is a parked vehicle, this may be as simple as plugging into the accessory port. However, newer vehicles tend to turn off the accessory port when the key is off.
eBay sells a car accessory socket attached to battery clamps. Use this with an external battery. Make sure you get one with a fuse.

A car battery will supply several hundred amps for 30 seconds.
An unfused short circuit results in wires glowing bright red and igniting anything flammable.

Car batteries, once fully discharged, will likely fail soon even if they take a charge. Material flakes off, and when that crud bridges between the plates, the battery is dead.
Deep-discharge batteries are designed to be run down and recharged fully, unlike car batteries.
Lithium 12-volt batteries are available which are direct replacements for lead-acid. They have built-in charge controllers. Compared to lead, these are easier on your back and harder on your wallet.
Solar panel and charge controller attached to a 12-volt battery can maintain a phone indefinitely.
Test results with a lead acid battery (measurements are 12-volt battery current):

Battery voltage 12.45V, Motorola g power phone, Samsung EP-LN920 car charger
Connected to Wi-fi, screen brightness turned down, phone fully charged, UnCloud audio on
UnCloud host running, not streaming, local video off: 170 - 200 mA
UnCloud host running, not streaming, local video on: 300 - 325 mA
Guest connected and viewing: 350 - 365 mA
This test shows the importance of using developer menu rather than local video to keep the phone awake.
With a 100 Ah battery, 10 days of observation or 20 days of standby is possible.

Using published amp-hour rating will slightly underestimate runtime, because the battery is more efficient at low drain.
Marine/RV battery "reserve capacity" rating is minutes at 25 amps, so divide reserve capacity by 2.4 to get amp-hours.
Samsung EP-LN920 car charger has an input voltage range of 11 - 36VDC. Two 12V batteries in series could be used. The charger's wide input voltage range would allow full use of the battery capacity.

UnCloud theory of operation - WebRTC

WebRTC is a protocol for direct communication between browsers, bypassing the server.
WebRTC was designed for voice and video, but can also carry arbitrary data.
WebRTC uses UDP with DTLS encryption. It gets past NAT using reciprocal UDP port mapping.
Normal operation of UDP NAT

Suppose a client has internal IP 192.168.1.2 and is behind a NAT with external IP 6.7.8.9
Client sends UDP from 192.168.1.2 port 56789 to public DNS server 8.8.8.8 port 53
NAT gateway maps 192.168.1.2 port 56789 to 6.7.8.9 port 45678 and forwards packet to 8.8.8.8 port 53
DNS on 8.8.8.8 port 53 replies to 6.7.8.9 port 45678
NAT remembers mapping and forwards reply to 192.168.1.2 port 56789
Most NAT gateways remember the exact mapping, so if A sends to B, B can reply along the same path.

WebRTC peer-to-peer operation over UDP NAT

Peer A has internal IP 192.168.1.2 and is behind a NAT with external IP 6.7.8.9
Peer B has internal IP 10.0.0.2 and is behind a NAT with external IP 9.8.7.6
Peer A sends UDP from 192.168.1.2 port 45678 to a public STUN server.
NAT maps 192.168.1.2 port 45678 to 6.7.8.9 port 56789 and relays to STUN server.
STUN server replies to A: "Your public address is 6.7.8.9 port 56789"
Peer B sends UDP from 10.0.0.2 port 12345 to the public STUN server.
NAT maps 10.0.0.2 port 12345 to 9.8.7.6 port 23456 and relays to STUN server.
STUN server replies to B: "Your public address is 9.8.7.6 port 23456"
A and B exchange their public IP and port information using a signaling service (explained later)
A starts sending UDP from 192.168.1.2 port 45678 to 9.8.7.6 port 23456
B starts sending UDP from 10.0.0.2 port 12345 to 6.7.8.9 port 56789
NAT on each side maps the path for replies to the outgoing packets.
The peers A and B can now communicate directly without further help from servers.
Peers should keep sending every few seconds to keep the NATs from dropping the paths.

STUN servers

STUN servers provide peers with their public IP address and port information.
They are low-resource and therefore freely available, most run by VOIP service providers.
STUN server requires two public IP addresses and a complex configuration.
It is easier to use public STUN servers rather than run your own.
UnCloud server gives each peer two STUN servers from a list of verified working ones.

TURN servers

Some firewalls do not permit direct peer-to-peer UDP connections.
A TURN server is a proxy that provides last-resort connectivity in this case.
TURN server is potentially high-bandwidth since it carries the encrypted traffic.
TURN servers require a username and password for access.
UnCloud server integrates with COTURN, creating a TURN account for each host user.
Accounts are deleted when the host record times out.
Guests use the credentials of the host they are contacting.
Known authenticated users could be provided a higher-bandwidth TURN server.

UnCloud server

The UnCloud server is written in Javascript and runs under nodejs.
UnCloud server must use HTTPS because the browser APIs for video and file access require it.
UnCloud server has built-in support for Lets Encrypt acme.sh .well-known directory.
Server delivers static HTML, Javascript, and other files via a fixed list in the server configuration block.
Static files will be sent with brotli or gz compression if .br or .gz files exist.
Server also provides a set of remote procedure calls which UnCloud browser client uses for connection setup.
Server saves its state periodically and on graceful exit (SIGINT, SIGTERM, SIGHUP).
State is reloaded at startup. This allows server restart without breaking user sessions.
Server is low-resource for small-scale use, runs well on a Raspberry Pi or very small (1GB) virtual machine.
Method for running a large environment (more than one UnCloud server) is described below.
Authentication strings associated with user IDs prevent trivial session hijacking.
Server RPCs: (all return 404 on failure)

GET /gethid

Returns a host-side ID, authentication string, pair of STUN servers, and TURN server with credentials.

GET /getgid?t=HOSTID

Returns a guest-side ID and authentication string. Host ID is only for load balancer.

GET /chkid?t=HOSTID
- Validates a host ID for connection.
- Returns STUN/TURN info for the host, or 404 if host not found.
GET /refid?t=ID&h=AUTH-HASH

Refreshes a guest ID so it does not expire. Called periodically by guests.

GET /getmsg?t=ID&h=AUTH-HASH

Receives signaling service messages. This is an Event Source.
Connection will remain open indefinitely and provide messages as they come in.
Idles are sent periodically. The ID will not expire as long as the client is connected.

POST /sendmsg?t=ID&h=AUTH-HASH

Sends a signaling service message. Message in post body.
Message format: f=FROM-ID t=TO-ID JSON-TEXT
Note: AUTH-HASH is that of the f= sender. t= in header must match t= in body.

UnCloud browser apps

UnCloud and UnCloud Watch are single-page web apps. Apparent page changes are done by switching divs on and off.
UnCloud apps communicate with each other by WebRTC, and with the server by AJAX calls and Event Source.
UnCloud uses https://github.com/feross/simple-peer and https://github.com/subins2000/simple-peer-files
Simple-peer-files has been slightly modified to accept custom STUN/TURN parameters and to stop an exception observed during testing.
Guest-to-host connection setup over signaling service

Host calls /gethid and obtains its ID, auth hash, and STUN servers.
Host calls /getmsg and waits for incoming WebRTC setup messages. Host displays its ID.
Guest calls /chkid with ID of host it wants to contact, receives STUN servers.
Guest calls /getgid and obtains its ID and auth hash.
Guest calls /getmsg to receive incoming WebRTC setup messages.
Guest sends !Uncloud:Init! to host via /sendmsg. This prepares a new WebRTC session.
Guest starts WebRTC as initiator. WebRTC handshake takes place via /sendmsg calls.
Once WebRTC is connected, UnCloud guest side can close Event Source.
UnCloud Watch guest keeps the Event Source open as it is required for media setup.

Guest-to-host protocol over WebRTC - UnCloud

Newly connected guest sends "login"
If no password or correct password, host responds "loggedin", if password required, host responds "passreq", if wrong password, host responds "passbad"
If "passreq" or "passbad", guest prompts for password and sends "login" with the password.
Once "loggedin" received, host sends "myname" if other than "host". Host sends either "nofiles" or "files" with list of offered files.
Host sends "resumekey" with a hash. This is used to resume a dropped connection even if the password has changed.
Guest also sends "myname" and "files" with its name and offered files if any.
Guest sends "shutdown" if window closed. Host sends "shutdown" on kick or window closed.
Either side sends "sendfile" with a number to retrieve an offered file. File send is carried out with simple-peer-files.
Either side sends "chat" with a message, and receives "chat" to display a message. Host sends "admsg" with a notification such as a user name change.
On reconnect after a drop, guest sends "resume" with a hash key and last seen message number, instead of "login"
All chat messages are relayed through the host. Host stores messages in an array. On resume, host sends all unseen messages to guest.
Host and guest send ping/pong keepalive back and forth to prevent any firewall timeout.
There are two different file download user experiences, depending on browser type and version.

Up-to-date browser uses window.showSaveFilePicker, so clicking Get displays a picker.
Downloads are queued and done one at a time. Multiple writers failed in testing.
This method has no file size limits. File over 20 GB has been tested.
Old browser uses window.URL.createObjectURL, so clicking Get does the download and then changes to Save.
This method is limited by RAM. File must be accumulated in memory before it can be saved.

Guest-to-host protocol over WebRTC - UnCloud Watch

Login, resume, myname, shutdown are same as UnCloud above.
On login, host sends "avoptions" with list of cameras, microphones, resolutions, and current settings for each.
Guest displays these and sends "sendvideo" with these current options. Host sends a video feed using simple-peer media.
If guest user changes an option, guest sends a new "sendvideo" with new selection. If settings have changed, host restarts video feed for all guests.
If host has battery API and monitor battery option is enabled, host sends "battery" which is displayed by guests.

How to run an UnCloud server

Setup your Linux or BSD server. A Raspberry Pi or small virtual machine works well.
Set your DNS A record and enable ports 80, 443, and 22 (ssh) on your firewall.
Install nodejs, brotli, gzip, and https://github.com/acmesh-official/acme.sh
sudo setcap cap_net_bind_service=+ep `which node`
- This lets your nodejs bind low ports without being root.
Unpack the UnCloud zip file.
brotli -k *.js *.html ; gzip -k *.js *.html
Install and configure coturn if you intend to provide a TURN server.
Edit uncloudserver.js and fill in homedir and TURN server. Create the wellknown_dir
Run: nodejs uncloudserver.js (https cert error will display)
Run the acme.sh command provided to fetch your Lets Encrypt certificate.
Restart the uncloud server, HTTPS should now come up.
Run your server in the background using uncloudserver.pl keepalive script, screen, or other means.

How to run a large multi-server environment

Handle a large load by subdividing the ID space. You can divide by 2, 5, 10 or multiples thereof.
UnCloud server startup options:

-http-port=PORTNO specifies alternate http port instead of port 80
-https-port=PORTNO specifies alternate https port instead of port 443
-savestate=FILENAME specifies alternate save state file
-prefix=CLIENTIDPREFIX restricts range of prefix IDs issued

0123456789 represent themselves
A = 01234, B = 56789
C = 01, D = 23, E = 45, F = 67, G = 89
-prefix=0 issues IDs starting with 0
-prefix=B issues IDs starting with 5, 6, 7, 8, or 9
-prefix=7E issues IDs starting with 74 or 75

Typical setup: nginx, haproxy, multiple UnCloud server processes.
Each UnCloud server process has its own internal port and save state file.
nginx handles HTTPS and static files (or use a CDN for .js and .mp3)
UnCloud servers handle RPCs. All UnCloud RPCs except /gethid have a t= URL parameter giving the ID of the server they should be sent to.
haproxy routes RPCs with a t= to the appropriate UnCloud server.
/gethid should be routed on a random or least-busy basis to one of the servers.
Guest sending /getgid?t= will get routed to the same server as its host, so no global UnCloud login database is required.
Provide multiple TURN servers if required, attached to the same authentication database. One will be assigned at random to each host.
Set cleanup_turn_frequency = null so servers don't delete each others' TURN accounts.
You will have to handle cleaning up stray TURN accounts. Read the TURN account list, pause for a few minutes, read the save state files, and delete any TURN account that is not in any one of them.
You should be able to handle any size workload with this method.

Open source

UnCloud uses https://github.com/feross/simple-peer and https://github.com/subins2000/simple-peer-files
UnCloud source code is available for download at
Developer contact: inglem atsign pobox dot com