Your device may have special abilities not covered by the current set of
traits
-- such as a "blink my light" trait for a device that can blink its lights. You
can define custom actions for your device that specify the commands sent to
your device to trigger special abilities.
To define a custom device action, you need the following:
- A pattern to match against the user query
- A custom device action to associate with a matched query
- Text spoken back to the user if the device supports the action
- A command name that is sent back to your device, along with any parameters
You create the custom device action by putting this information into an
action package
.
Action packages define the format for the Assistant responses. Unlike the
Actions SDK, custom device actions are fulfilled locally; you do not specify an
endpoint to process requests and provide responses. Custom device actions are
not conversational in nature.
Create an Action Package
Using the following as an example, create a file (such as
actions.json
) that
defines a test command: blinking an LED.
Copy it from the sample code you
downloaded in a
previous step
:
cd assistant-sdk-python/google-assistant-sdk/googlesamples/assistant/grpc/
cp ~/assistant-sdk-python/google-assistant-sdk/actions.json .
{
"manifest": {
"displayName": "Blinky light",
"invocationName": "Blinky light",
"category": "PRODUCTIVITY"
},
"actions": [
{
"name": "com.example.actions.BlinkLight",
"availability": {
"deviceClasses": [
{
"assistantSdkDevice": {}
}
]
},
"intent": {
"name": "com.example.intents.BlinkLight",
"parameters": [
{
"name": "number",
"type": "SchemaOrg_Number"
},
{
"name": "speed",
"type": "Speed"
}
],
"trigger": {
"queryPatterns": [
"blink ($Speed:speed)? $SchemaOrg_Number:number times",
"blink $SchemaOrg_Number:number times ($Speed:speed)?"
]
}
},
"fulfillment": {
"staticFulfillment": {
"templatedResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "Blinking $number times"
}
},
{
"deviceExecution": {
"command": "com.example.commands.BlinkLight",
"params": {
"speed": "$speed",
"number": "$number"
}
}
}
]
}
}
}
}
],
"types": [
{
"name": "$Speed",
"entities": [
{
"key": "SLOWLY",
"synonyms": [
"slowly",
"slow"
]
},
{
"key": "NORMALLY",
"synonyms": [
"normally",
"regular"
]
},
{
"key": "QUICKLY",
"synonyms": [
"quickly",
"fast",
"quick"
]
}
]
}
]
}
The previous example uses the following information to define the custom device
action:
- A pattern to match against the user query (
blink N times
)
- The custom device action to associate with a matched query (
com.example.actions.BlinkLight
)
for organizational purposes
- Text spoken back to the user if the device supports the action
(
Blinking N times
)
- A command name (
com.example.commands.BlinkLight
) that is sent back to
the device, along with any parameters (a number and possibly a description of the speed)
To define the query pattern, note the following:
- You can use
schema.org-defined types
in the query pattern.
- The
types [...]
array defines the list of custom types (for example,
$Speed
).
- You can use custom types in the query pattern. The user can speak any of the
synonyms in your custom type to match the query pattern.
- When a synonym does match, the type instance (
speed
) would
return the normalized key (
SLOWLY
). There can be multiple entities in
case, for example, there are different lights that support different speeds
of blinking.
- Parts of the request TTS pattern can be optional. For example, use
($Speed:speed)?
in the query pattern to make this part optional.
$
type
.raw
(for example,
$speed.raw
) in the
response TTS is replaced by the word(s) the user actually spoke.
For descriptions of many of these fields, see the
ActionPackage
documentation.
Deploy the Action Package
After you build your custom device action in an Action package, you can make the
Action package accessible to the Assistant.
While you can perform the steps in this section on your device, it may be easier
to do them on your development system. The following commands do not require a
virtual environment to run.
Download the
gactions
command line tool
.
Remove any existing credentials from the same directory as the
gactions
tool.
rm creds.data
Save your Action package to Google by using the
gactions
CLI.
Replace
project_id
with your Actions Console project
ID
.
./gactions update --action_package actions.json --project
project_id
The first time you run this command you will be given a URL and be asked to
sign in. Copy the URL and paste it into a browser (this can be done on any system).
The page will ask you to sign in to your Google account. Sign
into the Google account that created the project in a previous
step
.
After you approve the permission request from the API, a code will appear
in your browser, such as "4/XXXX". Copy and paste this code into the
terminal:
Enter the authorization code:
If authorization was successful, you will see a response similar to
the following:
Your app for the Assistant for project my-devices-project was successfully
updated with your actions.
Deploy your Action package into test mode by using the
gactions
CLI.
You must have saved your Action package to Google at least once before
running this command. Test mode enables the Action package on your
user account only.
./gactions test --action_package actions.json --project
project_id
Currently, you cannot test the project using the Actions simulator.
To update the Action package, use the
gactions update
command.
(Optional) You can create
localized Action packages
to support many
different languages and locales
at the same time in a single project.
Modify the sample
Do the steps in this section on the device.
nano pushtotalk.py
Add a handler for your custom action. Note that the handler below has already been added
to the sample code for the sample Action Package above.
...
device_handler = device_helpers.DeviceRequestHandler(device_id)
@device_handler.command('com.example.commands.BlinkLight')
def blink(speed, number):
logging.info('Blinking device %s times.' % number)
delay = 1
if speed == "SLOWLY":
delay = 2
elif speed == "QUICKLY":
delay = 0.5
for i in range(int(number)):
logging.info('Device is blinking.')
# GPIO.output(25, 1)
time.sleep(delay)
# GPIO.output(25, 0)
time.sleep(1)
Run the sample
Run the source code.
python pushtotalk.py
Try a query. For the example above, try the following:
Blink 5 times.
Note that the query needs to match the query pattern in the Action Package.