Using Dat to share files between two machiness

Dat is an awesome tool for the internet, and can be used to create some pretty cool applications.

Today we're going to utilise it to share files between two machines, for example from a computer to a mobile device.

In this article we're going to go through the basic setup, and then a later article we'll add functionality for a QR code that can be scanned from the second device, which can then be used to download the files.

To start, lets just get Dat working in the browser, for this we're going to utilise hyperdrive and dat-js, dat-js gives us a hyperdrive instance by default, which is pretty handy.

We're going to be utilising the create-react-app sandbox from Codesandbox.io to implement our project, you can follow along there, or utilise create-react-app on your own machine.

First we're going to need a way to create our storage, we're going to need to be able to do a couple of things:

Create a new archive
Load an existing archive
Forget an existing archive

So lets create an archive.js file, within this we'll add a few functions: get, create, list, and forget.

We're going to need a couple of little helper functions within this file so we can get and set items within localStorage as JSON strings:

js
function getLocal(key, def = undefined) {
  const value = localStorage.getItem(key);
  if (typeof value !== "string") {
    return def;
  }
  try {
    return JSON.parse(value);
  } catch (e) {
    return def;
  }
}

function setLocal(key, value) {
  localStorage.setItem(key, JSON.stringify(value));
}

Now within our list function, we're just going to return the value for the key "archives":

js
export function list() {
  return getLocal("archives", []);
}

For forget, we'll get our list, find the matching value, and then remove it:

js
export function forget(url) {
  const values = list();
  const foundIndex = values.indexOf(url);
  if (foundIndex === -1) {
    return values;
  }
  values.splice(foundIndex, 1);
  setLocal("archives", values);
  return values;
}

And then we'll also need a way to create a new archive, for this we'll first create our dat client:

js
// At the top of our file
import Dat from "dat-js";

const dat = new Dat();

Any instance of Dat happens to be an instance of an EventEmitter, and one of the events that it emits is archive, because of this we can listen for any archive that is created through this client and add it to our list of available archives:

js
dat.on("archive", archive => {
  const values = list();
  if (values.indexOf(archive.url) > -1) {
    return; // Already within the list of values
  }
  values.push(archive.url);
  setLocal("archives", values);
});

Now, we can create new archives by using our client, by default we want our files to persist on disk so that the data is always available on the machine once it is either uploaded or downloaded.

Because of our previous step, once the first write has occured to our new archive, the archive's url will be persisted between browser reloads. If no write occurs, you'll get a new archive every time, which is okay because nothing ever existed anyway.

js
export function create(options = { persist: true }) {
  return dat.create(options);
}

Lastly we will need a way to get archives that we already know the url for, if we didn't have the url contained in our list before, it will add it the same way as it does for create:

js
export function get(url, options = { persist: true }) {
  return dat.get(url, options);
}

We're going to need a way to keep track of this state within our React component as well, so we'll create a context where the provider initially loads the values from local storage, and then listens to additional changes, this will live within archives-context.js, notice how the state value contains list and forget:

js
import {
  createElement,
  createContext,
  useEffect,
  useState,
  useMemo
} from "react";
import { dat, list, forget } from "./archive";

export const Context = createContext({ list: [], forget: () => {} });

export const { Provider: BaseProvider, Consumer } = Context;

export function Provider({ children }) {
  const [value, setValue] = useState(list());
  const forget = useMemo(
    () => url => {
      setValue(forget(url));
    },
    [setValue]
  );
  useEffect(() => {
    const handler = archive => {
      if (value.indexOf(archive.url) > -1) {
        return;
      }
      setValue(value.concat(archive.url));
    };
    dat.on("archive", handler);
    return () => dat.off("archive", handler);
  }, [value, setValue]);
  return createElement(
    BaseProvider,
    { value: { list: value, forget } },
    children
  );
}

Next we'll create a context that we'll add our archive to, we'll create this in archive-context.js, this will use React's createContext method:

js
import { createContext } from "react";

export const Context = createContext(undefined);
export const { Provider, Consumer } = Context;

Now we'll create a component that will allow us to select an archive to perform actions against, this will be either by way of selecting a url, or inputting one directly, or creating a new archive, we'll create this within select.js:

js
import { Context as ArchivesContext } from "./archives-context";
import React, { memo, useContext, useState, useMemo } from "react";
import { Provider } from "./archive-context";
import { get as getArchive, create as createArchive } from "./archive";

export default memo(function SelectArchive({ children }) {
  const { list, forget } = useContext(ArchivesContext);
  const [url, setUrl] = useState(undefined);
  const [create, setCreate] = useState(false);
  const [textUrl, setTextUrl] = useState("");
  // Once url or create is set, it shouldn't ever change for this component
  const archive = useMemo(() => {
    if (url) {
      return getArchive(url);
    }
    return create ? createArchive() : undefined;
  }, [url, create]);
  if (archive) {
    return <Provider value={archive} children={children} />;
  }
  return (
    <div className="select">
      <ul>
        {list.length === 0 ? (
          <li>No archives available, please enter a url or create one</li>
        ) : (
          undefined
        )}
        {list.map(item => (
          <li key={item} onClick={() => setUrl(item)}>
            {item}
          </li>
        ))}
      </ul>
      <br />
      <br />
      <button type="button" onClick={() => setCreate(true)}>
        Create new archive
      </button>
      <br />
      <br />
      <input
        type="text"
        value={textUrl}
        onChange={event => setTextUrl(event.target.value)}
        placeholder="URL"
      />
      <button
        type="button"
        onClick={() => {
          setUrl(textUrl);
          setTextUrl("");
        }}
      >
        Continue
      </button>
    </div>
  );
});

Now, we can create a component that allows a user to upload multiple files, which will be then uploaded to dat, for this we will create a simple input field which accepts files, and a button which will submit them, this will upload them to dat and then reset the form. We'll create this within submit-file.js.

We're going to also utilise the usePromise method of react-use, so we'll need that as a dependency as well.

A couple of things read through for this component:

Before uploading, we're going to take our file objects, and turn them into instances of Uint8Array
After each upload, we want to know that these new files exist, so we'll emit a file event on our archive

js
import React, { memo, useState, useContext, useMemo, useRef } from "react";
import usePromise from "react-use/lib/usePromise";
import { Context } from "./archive-context";

export default memo(function SubmitFile() {
  const archive = useContext(Context);
  const mountedPromise = usePromise();
  const formReference = useRef(undefined);
  const [files, setFiles] = useState(undefined);
  const submit = useMemo(
    () => event => {
      event.preventDefault();
      if (!(files && files.length)) {
        return;
      }
      mountedPromise(async () => {
        await Promise.all(
          files.map(async file => {
            // We need a Uint8Array from our file
            const reader = new FileReader();
            const filePromise = new Promise((resolve, reject) => {
              reader.onload = resolve;
              reader.onerror = reject;
            });
            reader.readAsArrayBuffer(file);
            await filePromise;
            const arrayBuffer = reader.result;
            const array = new Uint8Array(arrayBuffer);
            await new Promise((resolve, reject) =>
              archive.writeFile(file.name, array, error =>
                error ? reject(error) : resolve()
              )
            );
          })
        );
      }).then(() => {
        // Create a new event for each new file
        files.forEach(file => archive.emit("file", file.name));
        setFiles(undefined);
        formReference.current.reset();
      });
    },
    [files, setFiles, archive, mountedPromise, formReference]
  );
  return (
    <form ref={formReference} onSubmit={submit}>
      <label>
        Files
        <input type="file" onChange={event => setFiles(event.target.files)} />
      </label>
      <br />
      <button type="submit">Submit</button>
    </form>
  );
});

Now that we have a way to upload files, we'll need a way to list whats available, this will be a mix of both listing our archive, and listening to our file event on the archive, this component will live in list-files.js:

js
import React, { memo, useMemo, useContext, useState } from "react";
import useAsync from "react-use/lib/useAsync";
import usePromise from "react-use/lib/usePromise";
import { Context } from "./archive-context";

export default memo(function ListFiles() {
  const archive = useContext(Context);
  const readdir = useMemo(
    () => () =>
      new Promise((resolve, reject) =>
        archive.readdir("./", (error, list) =>
          error ? reject(error) : resolve(list)
        )
      ),
    [archive]
  );
  const [promise, setPromise] = useState(readdir());
  const state = useAsync(useMemo(() => () => promise, [promise]);
  const reload = useMemo(
    () => () => {
      setAdditionalFiles([]);
      setPromise(readdir());
    },
    [readdir, setPromise]
  );
  const [additionalFiles, setAdditionalFiles] = useState([]);

  const files = useMemo(() => (state.value || []).concat(additionalFiles), [
    state.value,
    additionalFiles
  ]);

  const mountedPromise = usePromise();
  const download = useMemo(
    () => file => {
      return () => {
        mountedPromise(async () => {
          const array = await new Promise(resolve =>
            archive.readFile(file, {}, (error, data) =>
              resolve(error ? undefined : data)
            )
          );
          if (!array) {
            return alert("Failed to download file");
          }
          const blob = new Blob([array]);
          const url = URL.createObjectURL(blob);
          window.open(url, "_blank");
        });
      };
    },
    [mountedPromise, archive]
  );
  return (
    <React.Fragment>
      <button type="button" onClick={reload}>
        Reload
      </button>
      <ul className="files">
        {state.error ? <li>Error: {state.error}</li> : undefined}
        {state.loading ? <li>Loading</li> : undefined}
        {files.map((file, index) => (
          <li key={index} onClick={download(file)}>
            {file}
          </li>
        ))}
      </ul>
    </React.Fragment>
  );
});

Now we can put the two together, and make a list & upload component, we're going to call this manage-files.js:

js
import React, { memo } from "react";
import ListFiles from "./list-files";
import SubmitFile from "./submit-file";

export default memo(function ManageFiles() {
  return (
    <React.Fragment>
      <SubmitFile />
      <ListFiles />
    </React.Fragment>
  );
});

Now, all we should need to do is render our SelectArchive component, with the child ManageFiles within index.js:

js
import React from "react";
import ReactDOM from "react-dom";

import "./styles.css";

import SelectArchive from "./select";
import ManageFiles from "./manage-files";

function App() {
  return (
    <div className="App">
      <SelectArchive>
        <ManageFiles />
      </SelectArchive>
    </div>
  );
}

const rootElement = document.getElementById("root");
ReactDOM.render(<App />, rootElement);

Now we have a way to upload files to dat, and list what is available.

In the next article we're going to cover sharing a dat archive with other devices.

Fabian Cook

Software Engineer @ Dovetail

JavaScript Developer.

Read similar articles

How to Build and Deploy Superheroes React PWA Using Buddy React Quickstart For Beginners

Sep 24, 2020

React

Using Dat to share files between two machiness

Start for free

Self hosted

Cloud