Voice Assistant Guide

UltimateSensor v2 includes a built-in microphone, speaker and standard local wake word support for local Home Assistant voice assistant use.

Important: This guide is for the regular UltimateSensor v2. The UltimateSensor Mini v2 does not include a microphone or speaker and cannot be used as a voice-assistant satellite.

Choose Your Voice Setup

UltimateSensor v2 supports two separate voice setups:

Both setups use the local Okay Nabu wake word. The firmware decides which setup you have: local firmware uses Home Assistant Voice, cloud firmware uses SmartHomeShop Cloud Voice.

SmartHomeShop Cloud Voice

Cloud Voice is intended for users who do not have a Home Assistant voice pipeline. It requires compatible UltimateSensor v2 cloud firmware, an internet connection and a device claimed in the SmartHomeShop App.

1. Open Settings -> Cloud Voice in the SmartHomeShop App.
2. Enable Cloud Voice.
3. Select the response voice. The assistant answers in the language you speak.
4. Optionally enable conversation memory, hands-free clarifications and live internet search.
5. Say Okay Nabu, wait for the listening indication and ask your question.

The wake word remains local. After activation, the spoken request is sent securely to SmartHomeShop Cloud for speech recognition, answer generation and speech output. The answer is returned to the UltimateSensor speaker.

Follow-up Questions

When conversation memory is enabled, Cloud Voice remembers up to six recent question-and-answer turns for ten minutes.

If Hands-free clarification questions is enabled and the assistant cannot answer correctly without one essential detail, it asks one short question and automatically listens for your reply. You do not need to say Okay Nabu again. The listening window closes after eight seconds of silence, and the assistant stops after at most two clarification rounds. Normal new questions and optional follow-up questions still require Okay Nabu.

Disable conversation memory to handle every request independently. This also disables hands-free clarifications because the assistant would otherwise lose the context of its own question.

Current Information and Internet Search

When Search the internet for current answers is enabled, Cloud Voice can automatically search for information that may have changed, including:

• opening hours and nearby restaurants;
• events, news and weather;
• prices, public transport and other schedules.

Timeless questions normally do not trigger a search. Only searches actually performed by the assistant count toward the separate daily and monthly internet-search allowance shown on the Voice settings and pricing pages. Source links are stored with the request and can be reviewed by SmartHomeShop administrators, but URLs are not read aloud.

Current information can still be incomplete or outdated. Confirm important opening hours, prices, travel information and availability with the original provider.

Nearby Questions and Device Location

For questions such as “Which restaurant near me is open?”, link the UltimateSensor to a location in the SmartHomeShop App. Cloud Voice then receives:

• the location's city and country;
• latitude and longitude rounded to roughly 100 metres.

The street name and house number are not sent as AI search context. Location context is used only when the question is explicitly local or asks for something nearby. Without a linked location, the assistant asks you to link one or name a city instead of guessing.

Cloud Voice Controls

The Voice settings page lets you:

• turn Cloud Voice on or off;
• select a female or male response voice;
• turn short-lived conversation memory on or off;
• turn hands-free clarification questions on or off;
• turn live internet search on or off;
• view the separate Voice request and internet-search usage limits;
• configure your own OpenAI API key on supported paid plans.

Set the speaker volume separately for each UltimateSensor v2 under Devices -> Edit -> Speaker. A saved volume is restored after a restart and applies to Cloud Voice, spoken notifications and other device audio.

Cloud Voice and internet-search limits depend on the subscription plan. A personal OpenAI API key changes who pays the OpenAI processing costs, but plan safety limits remain active. When the limit is reached, the UltimateSensor tells you with a spoken message.

What You Need

• UltimateSensor v2
• Home Assistant with the ESPHome integration
• A configured Home Assistant voice pipeline
• For a fully local voice pipeline: Whisper and Piper, or equivalent Home Assistant voice components. The wake word already runs on the device.

Set Up Home Assistant Voice

1. In Home Assistant, go to Settings -> Voice assistants.
2. Create or edit an assistant.
3. Choose your conversation agent.
4. Choose speech-to-text and text-to-speech options. The UltimateSensor handles the wake word itself.
5. Save the assistant.

For a local setup, follow Home Assistant's voice documentation: Home Assistant voice control.

Select the Assistant Pipeline on UltimateSensor v2

1. Go to Settings -> Devices & services -> ESPHome.
2. Open your UltimateSensor v2 device.
3. Find the assistant pipeline select entity.
4. Select the Home Assistant voice assistant you configured.
5. Check that the Use Wake Word switch is on. It is on by default.

Entity names can differ depending on your device name and firmware version, but they are usually grouped under the UltimateSensor v2 device in Home Assistant.

Speaker, TTS and Alerts

UltimateSensor v2 exposes a media player entity for audio output. You can use it for:

• Text-to-speech announcements
• Alarm sounds
• Voice assistant responses
• Local audio feedback from automations

Example TTS automation:

service: tts.speak
target:
  entity_id: media_player.ultimatesensor_v2_media_player
data:
  message: "CO2 levels are high. Please ventilate the room."

Replace the entity id with the media player entity shown for your UltimateSensor v2.

LED Indications

The three front LEDs show what the voice assistant is doing:

At startup the LEDs breathe white for about 10 seconds with a startup sound. Turn this off with the Boot Sound & Animation switch.

Troubleshooting

Wake Word Does Not Trigger

• Confirm the Use Wake Word switch is enabled. On cloud firmware this switch is called Cloud Voice.
• Say Okay Nabu. This is the only wake word in the firmware and it runs on the device, so the pipeline needs no wake word engine.
• Make sure the device is not too far away or placed near constant noise.
• Restart the UltimateSensor v2 and the Home Assistant voice apps.

No Audio Output

• Check the UltimateSensor v2 media player entity in Home Assistant.
• Verify the media player volume.
• Test with a short TTS message before using longer announcements.
• Check ESPHome logs for I2S or media player errors.

Assistant Hears You But Does Not Respond Correctly

• Check the selected language in the Home Assistant assistant pipeline.
• Confirm speech-to-text and text-to-speech services are running.
• Try a shorter command.

Privacy

With Home Assistant Voice, processing is handled by your Home Assistant setup and can run fully on your local network. With Cloud Voice, audio, transcripts, answers, limited conversation context and optional approximate location context are processed in SmartHomeShop Cloud. The Voice Improvement Program is on by default: we then keep the microphone recording, transcript, answer and generated speech for at most 30 days, and authorized SmartHomeShop administrators may listen to and review them. You can switch this off at any time under Settings -> Cloud Voice, which also deletes the stored data. Review the SmartHomeShop App privacy settings and privacy policy before enabling Cloud Voice.

Related

• Hardware Versions
• Usage Guide
• Troubleshooting
• Home Assistant voice control