new flow - possible?

scocz

any way to create a flow where homey replies you with "i couldnt understand you" when it gets a command it doesnt know and just executes when it does recognize it?

novaflash

I have looked in my Homey but I can't find such an option. I don't think that option exists. I would like a card option that says "When: Homey didn't understand you".

scocz

novaflash said:

I have looked in my Homey but I can't find such an option. I don't think that option exists. I would like a card option that says "When: Homey didn't understand you".

+1

Sommo

+1
because now, it turns to rainbow, i think it could be very usefull know haveing negative feedback, so homey could stay on listening mode, without having to trigger it again

Phuturist

Homey is not a human that knows when it doesnt understand you. The speech to text service just translates what it thinks it heard but there is no way for it to tell if it heard you correct and/or the translation service was correct. This is not gonna happen guys.

Sommo

do you use siri? if so you will know that if what u asked is not comprensible or it dont understend you, it will give you that feedback...

scocz

Phuturist said:

Homey is not a human that knows when it doesnt understand you. The speech to text service just translates what it thinks it heard but there is no way for it to tell if it heard you correct and/or the translation service was correct. This is not gonna happen guys.

When I say "ok homey blablabla" it should reply me with "im sorry i dont understand what you mean". just attach things homey already knows to actions and all the other things to this sentence

bvdbos

@Phuturist A lot of times Homey listens, sends audio to the webservice and doesn't get a reply back (I have a parrot flow running). I think it's usefull for these cases...

Bumblez

Hmm... don't have my Homey yet, so I can't test this, but is it possible to create a time-based flow that triggers when the current time > (value of a variable + 5 seconds)?
So:
- when Homey starts listening, set a variable to the current time
- when Homey recognizes something/triggers an action, set the variable to 0.
- if the variable is set, and the current time > value of the variable + 5 seconds, then say "I'm sorry, I don't understand what you mean"  ?

JohanKuster

Bumblez said:

Hmm... don't have my Homey yet, so I can't test this, but is it possible to create a time-based flow that triggers when the current time > (value of a variable + 5 seconds)?
So:
- when Homey starts listening, set a variable to the current time
- when Homey recognizes something/triggers an action, set the variable to 0.
- if the variable is set, and the current time > value of the variable + 5 seconds, then say "I'm sorry, I don't understand what you mean"  ?

When does homey recognize something? Some commands are internal ones, some are defined by the user. Homey does probably know when it has no match, we don't. You need a voice trigger that fires when no match was found. you would probably also want a label to be used in the 'Ask confirmation' condition where you can repeat what homey received as input like 'Sorry, I did not understand you. Did you mean :'[label].  Then you want some way to process the NO answer to this question. All not possible at the moment without using a geek solution.  I proposed several enhancements for flow management but no reponse yet from Athom.

Bumblez

@JohanKuster  ;; you're right, there's too many things in that flow that are not supported yet. That's why I was trying an alternative approach (until Athom actually adds a 'Homey did not recognize a text' trigger), not as much focused on the concept that Homey actually responds to not recognizing a text, but more around the situation where the follow-up to "Ok Homey" did not result in any flows being triggered... that's why I proposed the time-based approach. Maybe my approach could work using the CountDown timer app?

JohanKuster

@Bumblez : I did something like that mentioned in another topic

Ok, this should work

Flow1 : when [Motion sensor triggered] then  [start Countdown1 1 sec] + [start Countdown2 10 sec] 
Flow2 : when [Countdown1] and [Confirmation : "Are you Ruben?"]  then ["Greetings Ruben"] + [stop Countdown2]
Flow3 : when [Countdown2] then ["warning"] + [Play a sound "Alarm.wav"]

Now I have to teach the dog and cats to say yes 

Phuturist

BasVanDenBosch said:

@Phuturist A lot of times Homey listens, sends audio to the webservice and doesn't get a reply back (I have a parrot flow running). I think it's usefull for these cases...

True, but that is a different use case as it's easier to check on no response then the right/expected response.

I currently have the issue with no response about 95% of the time I talk to Homey currently (started with 0.8.29 and making speech useless, before my voice recognition was pretty good). I'm still not sure what is happening, either the voice sample is not send to Athom's processing server at all or the speech to text services do not give a reply. It would be nice to know in both cases that Homey was unable to process the speech to text (once again, something that is different than misunderstood the text).

DaneeDeKruyff

@Phuturist

It's pretty easy to know if it's the right/expected response; If the response that comes back does not trigger anything at all in homey, homey doesn't know what to do with it -> "I'm sorry I don't understand you"

Bumblez

Ok, so this should already be possible:

Flow1 : when [Homey starts listening] then [set 'stillListening' global variable to 1] + [start Countdown1 5 sec]
Flow2 : when [Countdown1] and ['stillListening' variable = 1] then ["I'm sorry, I don't understand what you mean""] + [set 'stillListening' variable to 0]
(all other flows:) [set 'stillListening' variable to 0]

That '5 sec' may have to be tweaked a bit, also depending on the speed of the speech recognition servers...

Phuturist

DaneeDeKruyff said:

@Phuturist

It's pretty easy to know if it's the right/expected response; If the response that comes back does not trigger anything at all in homey, homey doesn't know what to do with it -> "I'm sorry I don't understand you"

True, I was overthinking this (not sure how this would work with built in commands though as this does no trigger a flow).  

JohanKuster

Bumblez said:

Ok, so this should already be possible:

Flow1 : when [Homey starts listening] then [set 'stillListening' global variable to 1] + [start Countdown1 5 sec]
Flow2 : when [Countdown1] and ['stillListening' variable = 1] then ["I'm sorry, I don't understand what you mean""] + [set 'stillListening' variable to 0]
(all other flows:) [set 'stillListening' variable to 0]

That '5 sec' may have to be tweaked a bit, also depending on the speed of the speech recognition servers...

You will have to test this for internal commands. I wonder.

Bumblez

JohanKuster said:

You will have to test this for internal commands. I wonder.

Good point... I hadn't thought about the fact that internal commands don't have a separate Flow entry, thus you can't add the [variable to 0] there... we'd need some way to catch and reroute all internal commands, so
Flow: when [internal command is recognized] then [set 'stillListening' variable to 0] and [trigger the original internal command]  (preferably without having to set this up for every single internal command)

... which I guess is not possible yet.

TedTolboom

I would propose to introduce / enable this only when the false triggering issue has been solved... otherwise we are adding a (currently quit frequent) "sorry I didn't understand you" in addition or as replacement to the "Oops I couldn't reach the server"...

JohanKuster

 So, can we let homey listen to itself? Throw in some deep learning and all problems are solved.

aloft

JohanKuster said:

 So, can we let homey listen to itself? Throw in some deep learning and all problems are solved.

42 ! 

JohanKuster

@aloft : I think some kind of 42 app should be produced.