Self-Host VoiceMem and Access It with Localtonet

Why self-host the VoiceMem interactive demo?

Voice agents often need more than a transcript of the current conversation. A useful long-running agent may need to retrieve facts, recognize recurring preferences, and account for emotional or relationship context without repeatedly injecting an entire conversation history into the model. VoiceMem approaches this problem as a dedicated memory layer for real-time spoken interaction.

The project divides memory into two cooperating areas. Its informational “left brain” organizes factual memories through schema and entity structures. Its emotional “right brain” represents persona, emotion, and relationship-oriented information. VoiceMem also provides streaming interfaces intended to begin memory work while a person is still speaking, rather than waiting for a complete turn before starting retrieval.

Self-hosting the repository’s interactive web demo is useful when you want to evaluate that workflow on your own machine, inspect its behavior, observe its logs, or demonstrate it to a remote collaborator. The documented demo runs as a local HTTP service at http://localhost:8787. That local address is intentionally reachable from the same machine, but it is not automatically reachable from another network.

With Localtonet, we can publish that working local HTTP service through a public HTTPS address. Our client establishes an outbound connection to a Localtonet relay, so the workflow does not require inbound router port forwarding, a public IP address, VPN setup, or a new inbound firewall rule. The tunnel remains available only while the selected Localtonet client is connected and the tunnel is running.

How VoiceMem fits into a voice-agent architecture

VoiceMem is a memory component rather than a complete hosted voice-agent platform. Its job is to ingest information, structure memory, retrieve relevant context, and make that context available to another model or application. The repository provides library interfaces, examples, evaluation code, model-related tools, and an interactive web demo that makes experimentation easier.

Informational memory

The left-brain side manages factual information. VoiceMem describes a schema and entity organization that helps route and rank stored memories. When an agent receives a new query, the system can retrieve a small set of relevant memories instead of passing an entire historical transcript into the responding model.

Emotion and persona memory

The right-brain side is intended to preserve information about emotion, persona, and relationships. This distinction matters because a factual statement and an emotional reaction are not always useful under the same retrieval rules. A user’s preference, identity-related detail, or response pattern can require different treatment from a simple factual event.

Streaming behavior

The project includes a streaming interface that can process incoming audio in chunks. Its documented example uses voice activity detection, partial transcripts, and speculative memory retrieval while a speaker is still talking. At the end of the turn, the application can consume the retrieved memory and decide whether the completed utterance should be ingested.

This guide does not reproduce or extend that programmatic integration. Our goal is narrower and more practical: install the documented package and model assets, start the repository’s interactive demo, verify the HTTP endpoint, and then expose that endpoint through an HTTP tunnel.

Prerequisites and planning

Complete the installation on the machine that will run VoiceMem. The Localtonet client must eventually run on that machine or on another device that can reach the VoiceMem service over the local network. Keeping both processes on the same machine is the simplest arrangement because the tunnel can target the loopback address directly.

The documented installation commands require Git, Python, pip, internet access to obtain Python packages, and the Hugging Face command-line tool used to download the default models. You also need enough storage and memory for the packages and downloaded model assets. The supplied VoiceMem documentation does not establish a universal minimum Python version, operating-system matrix, CPU requirement, GPU requirement, memory requirement, or model download size for this exact demo workflow.

Check the following before starting:

• Git is installed and available as git.
• Python is installed and available as python.
• The Python environment can run pip.
• The machine can access the Python package index, GitHub, and Hugging Face.
• You have permission to create files in the directory where the repository will be cloned.
• You have sufficient free disk space for the repository, Python dependencies, and model files.
• Port 8787 is not already occupied by another local process.
• You can install and run the Localtonet client when you are ready to add remote access.

Do you need the optional SLM package?

No, not for the basic documented installation. The project lists an optional slm extra for its fine-tuned Qwen response model. Install that extra only if you specifically plan to use the corresponding response-model functionality. Adding optional model dependencies unnecessarily can increase installation time and complicate troubleshooting.

Will VoiceMem need external credentials?

The library’s documented basic-ingestion example accepts an OpenAI key for fact extraction during memory writes, while memory retrieval is described as local. However, the supplied startup documentation for python web/run.py does not fully establish every provider setting or credential that a particular demo interaction may require. Do not place secrets directly in source files, shell history, screenshots, tunnel URLs, or browser-visible fields unless the project explicitly requires that location.

Start the demo from a clean terminal and read any configuration error it prints. If a selected feature requires an external provider, configure that provider using the current VoiceMem documentation and your organization’s secret-management practices. This guide does not invent an environment variable for the web demo when the evidence does not establish one.

Install VoiceMem and download its models

The official quick-start sequence clones the repository, enters its directory, installs the published voicemem Python package, installs the Hugging Face client, and downloads the default model collection into ./models. Cloning is essential for this tutorial because the pip package contains the library, while the interactive demo lives in the repository.

git clone https://github.com/xzf-thu/VoiceMem.git

cd VoiceMem

pip install voicemem

pip install -U huggingface_hub
hf download zhifeixie/VoiceMem_Default_Models_Env --local-dir ./models

Do not move the downloaded files or substitute a different directory unless the current VoiceMem documentation explicitly explains how to change model discovery. The documented command places the assets under ./models, relative to the repository root.

Optional SLM installation

If you specifically need the project’s optional fine-tuned Qwen response-model support, install the documented extra:

pip install "voicemem[slm]"

This optional command is not a replacement for the normal package installation and does not remove the need to download the required default models. For initial troubleshooting, it is usually easier to complete the basic installation and start the demo before introducing optional components.

Confirm that the download completed

The Hugging Face command should return to the prompt without a fatal error. Confirm that a models directory now exists beneath the VoiceMem repository. The project evidence supplied for this guide does not specify a definitive file count or total byte size, so do not use an invented number as a health check.

If the download stops partway through, preserve the terminal output. Typical categories to investigate include insufficient storage, interrupted network access, inability to resolve the remote host, access restrictions imposed by a proxy, and a missing or outdated hf command. Resolve the underlying error before trying to run the demo.

Start the VoiceMem interactive web demo

Run the demo from the root of the cloned VoiceMem repository. This working directory matters because the startup script and model directory are referenced through repository-relative paths.

python web/run.py

http://localhost:8787

Keep the terminal visible during the first launch. Model initialization can require more time than an ordinary static web page, and startup failures are usually easier to diagnose from the process output than from the browser alone. Do not configure a public tunnel while the terminal is still reporting a fatal exception or repeatedly restarting.

Understand the default file logging

The demo saves a copy of terminal output by default, including Python logging and Uvicorn output. The documented log filename follows this pattern:

results/logs/voicemem-time-PID.log

Each line includes a timestamp and an indication of whether it came from standard output or standard error. The startup terminal prints the actual log path, which is more reliable than trying to predict the generated filename.

To select a specific log file, use the documented option:

python web/run.py --log-file results/logs/debug.log

To disable file logging temporarily, use:

python web/run.py --no-file-log

Use only one startup mode at a time. A dedicated debug log can be helpful when reproducing a problem, while disabling file logging may be appropriate when you do not want potentially sensitive interaction output retained on disk.

Verify VoiceMem locally before creating a tunnel

Local verification separates application problems from tunnel problems. If http://localhost:8787 does not work on the host machine, a public tunnel cannot repair the VoiceMem process. It can only forward traffic to the target you configure.

1. Confirm that the process stays running

After executing python web/run.py, the command should remain active rather than immediately returning to the shell because it is serving the web application. If it exits, scroll to the first meaningful exception and inspect the generated log when available.

2. Load the exact documented endpoint

Open http://localhost:8787 on the same machine. Use the explicit http:// scheme for this local test. If the browser rewrites the URL, clear the address and enter it again exactly.

3. Confirm that the page is functional, not merely reachable

A successful TCP connection or a page shell is only the first layer of verification. Confirm that the interface renders and that its expected controls can be used. Watch the VoiceMem terminal while interacting with the page. Application errors that occur after initial page load may appear only when a model or audio feature is first invoked.

4. Review browser permissions

If you test microphone-dependent functionality, the browser may request microphone permission. Grant access only to the origin you intend to use and only when required for the test. Browser permission behavior can differ between local HTTP origins and public HTTPS origins, so repeat the functional check after creating the tunnel.

5. Establish a clean baseline

Record what works locally before adding remote access. At minimum, note whether the page loads, whether the relevant interactive action completes, whether the terminal reports an exception, and whether stopping and restarting the process restores the same behavior. This baseline prevents a model-loading problem from being mistaken for a Localtonet configuration problem.

Expose the working VoiceMem demo with a Localtonet HTTP tunnel

Once the local browser test succeeds, you can configure remote access. An HTTP tunnel is the appropriate Localtonet tunnel family for this workflow because VoiceMem provides a browser-facing HTTP service. You do not need a raw TCP tunnel merely to open the web interface.

Our client creates an outbound connection to a relay server and forwards requests from the assigned public address to the local target. For this setup, the target is the VoiceMem service on the loopback address and port 8787.

If the Localtonet client runs on another device, 127.0.0.1 would refer to that other device rather than the VoiceMem host. In that arrangement, the tunnel target must be an address and port reachable from the Localtonet client device. VoiceMem’s supplied demo instructions establish the localhost endpoint, but they do not establish a documented LAN-listening configuration. Do not invent a bind-address flag. Running both components on the same machine avoids that uncertainty.

For the current product workflow and installation options, consult our Localtonet documentation.

Security boundaries for remote VoiceMem access

A public HTTPS URL protects the browser-facing transport at the tunnel edge, but it does not turn an unauthenticated demo into an authenticated application. Transport security and application authorization solve different problems. The project documentation supplied for this guide does not confirm a built-in login system, role model, rate control suitable for production, hardened session management, or tenant isolation for the interactive demo.

Assume that anyone who obtains a reachable public URL may attempt to load it. Voice memory can contain sensitive information, including transcripts, personal preferences, emotional context, inferred relationships, voice-related data, and provider responses. A demo populated with real personal data therefore creates a much higher exposure risk than an empty test instance.

Use synthetic data during remote testing

Test with invented names, fictional preferences, and disposable audio rather than real customer conversations. Avoid uploading health information, financial information, confidential business discussions, biometric material, or private relationship details. Synthetic test data reduces the impact of accidental exposure and makes logs easier to share internally.

Keep exposure temporary

Start the tunnel immediately before the remote test and stop it as soon as the session ends. Do not leave the demo running publicly because it might be useful later. The ability to stop a Localtonet tunnel independently from deleting it makes short test windows practical.

Protect tokens and provider credentials

A Localtonet device token identifies the client device and must remain secret. External model-provider credentials also require protection. Never insert either type of secret into a public URL, browser query string, shared screenshot, issue report, or committed configuration file. If a credential is accidentally exposed, revoke or rotate it through the system that issued it.

Apply access controls before production use

If your intended deployment requires public or multi-user access, place a properly designed application layer in front of the memory engine. That layer should enforce authentication, authorization, input validation, session handling, request limits, audit policy, secret management, and data-retention controls appropriate to your environment. These controls are application responsibilities and are not proven by the fact that the demo can be reached through HTTPS.

Common operations after setup

Starting a new session

Start VoiceMem first and confirm the local URL. Then run the Localtonet client, verify that the intended device is connected, and start the existing HTTP tunnel. This order ensures that the public endpoint does not begin forwarding to an unavailable local service.

Stopping access

Stop the Localtonet tunnel to remove the public path. Then stop the VoiceMem process in its terminal. Stopping only VoiceMem leaves the tunnel running but without a healthy target. Stopping only the tunnel removes public access while preserving the local demo.

Restarting after package or model changes

Stop the demo cleanly, apply the intended change, and start it again from the repository root. Verify locally before restarting the tunnel. If an update changes dependencies, model layout, command-line options, or the demo port, follow the current upstream release documentation rather than assuming this guide’s commands still describe the newer release.

Managing logs

Use the default timestamped log when you want a history of each launch. Use --log-file when reproducing a specific problem and you want a known output path. Use --no-file-log when persistent logs are undesirable. Regardless of the mode, establish a retention and deletion practice if the demo processes sensitive content.

Changing the public address

HTTP and File Server tunnels can use a random subdomain, a supported custom subdomain, or a custom domain. These Process Types serve the same content through a public HTTPS address. Availability can vary by current configuration or plan, and exact custom-domain DNS requirements must be taken from the current Localtonet dashboard or documentation.

Troubleshooting VoiceMem and Localtonet access

Troubleshoot from the inside out. Start with the Python process, then the localhost endpoint, then the Localtonet client, and finally the public URL. This sequence isolates each layer and avoids changing a tunnel configuration to compensate for an application error.

Installation succeeds in one terminal but startup fails in another

This commonly indicates that the two terminals are using different Python environments or different command search paths. Compare how Python and pip are resolved in each environment according to your operating system’s standard tooling. Reopen the environment where the package was installed, return to the repository root, and start the demo there.

The model download appears incomplete

Review the output from the hf download command rather than relying only on the existence of a models directory. A directory may have been created before the transfer failed. Resolve storage, network, proxy, or permission errors, then run the documented download command again.

The public URL returns an upstream or connection error

First reload http://localhost:8787 on the VoiceMem host. If it fails, restart or repair VoiceMem. If it succeeds, verify that Localtonet is running on the same machine when the target is 127.0.0.1. Then confirm that you selected the correct device and started the tunnel after creating it.

The page works locally but not from another machine on the LAN

The documented endpoint is localhost. Localhost is scoped to the machine running the process, so direct LAN access is not established by the supplied VoiceMem instructions. This does not prevent a Localtonet client on the same host from reaching the loopback service. Do not change VoiceMem’s bind behavior using an undocumented flag merely to make LAN access work.

Frequently asked questions