Presence detection

MatjaLipu

I've created an app which supports alternative presence detection.
It supports ARP scan, ping, TCP connect and HTTP signaling.

• App store
• Source
  

Due Homey security restrictions, this app requires presence app running on other device (such as Raspberry Pi), to collect presence data.

Steps to start using app:

1. Install standalone presence app
2. Configure presence app (see readme)
   
3. Install Homey app
   
4. Configure Homey app (settings page) set URL to presence app

Common issues:
- Configure CORS in presence app config, so Homey app is able to communicate with presence app. Add Homey address (IP) to "allowedOrigins". For example "http://192.168.13.37".
- in Homey app settings, URL must be without trailing slash.

Phuturist

This is great stuff. I currently use Tasker on our Android phones which gets this job done as well but will look into this when I have some more time. Gonna try to get the stand alone app running on my Synology NAS as this is running 24/7 anyway.

MatjaLipu

The whole point of this app is not to require any app (Homey or Tasker) on device being detected. If device running standalone app have BLE it could also detect iBeacon devices.

AndyWilde

I'm having a problem installing this,  but to be fair I am not all that familiar with using a Raspberry Pi.

I have installed Node and npm

The error I get when I install is as follows

npm ERR! Linux 4.1.18+

npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "start"

npm ERR! node v4.2.1

npm ERR! npm  v2.14.7

npm ERR! code ELIFECYCLE

npm ERR! presence-tracker@1.1.0 start: `babel-node bin/presence`

npm ERR! Exit status 1

npm ERR! 

npm ERR! Failed at the presence-tracker@1.1.0 start script 'babel-node bin/presence'.

npm ERR! This is most likely a problem with the presence-tracker package,

npm ERR! not with npm itself.

npm ERR! Tell the author that this fails on your system:

npm ERR!     babel-node bin/presence

npm ERR! You can get their info via:

npm ERR!     npm owner ls presence-tracker

npm ERR! There is likely additional logging output above.

If you could post the steps needed to get this to work on a vanilla raspberryPi it would be a great help

MatjaLipu

Added instructions how to install node on Raspberry Pi.

Phuturist

I got a similar error as above when trying to start the server on my Synology NAS. I followed similar instructions as for the Raspberry PI but installation has thrown the following warnings.

root@Diskstation:/volumeUSB1/usbshare/Presence# npm install
npm WARN package.json presence-tracker@1.2.0 No repository field.
npm WARN unmet dependency /volumeUSB1/usbshare/Presence/node_modules/babel-cli/node_modules/bin-version-check/node_modules/bin-version/node_modules/find-versions/node_modules/meow/node_modules/read-pkg-up/node_modules/find-up requires path-exists@'^2.0.0' but will load
npm WARN unmet dependency /volumeUSB1/usbshare/Presence/node_modules/babel-cli/node_modules/path-exists,
npm WARN unmet dependency which is version 1.0.0

Then when starting the server I get the following:

root@Diskstation:/volumeUSB1/usbshare/Presence# npm start

> presence-tracker@1.2.0 start /volumeUSB1/usbshare/Presence
> babel-node bin/presence

sh: babel-node: command not found

npm ERR! Linux 2.6.32.12
npm ERR! argv "node" "/usr/local/bin/npm" "start"
npm ERR! node v0.10.41
npm ERR! npm  v2.15.1
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR! presence-tracker@1.2.0 start: `babel-node bin/presence`
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the presence-tracker@1.2.0 start script 'babel-node bin/presence'.
npm ERR! This is most likely a problem with the presence-tracker package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     babel-node bin/presence
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs presence-tracker
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!
npm ERR!     npm owner ls presence-tracker
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /volumeUSB1/usbshare/Presence/npm-debug.log

And the npm debug log gives the following information:

0 info it worked if it ends with ok
1 verbose cli [ 'node', '/usr/local/bin/npm', 'start' ]
2 info using npm@2.15.1
3 info using node@v0.10.41
4 verbose node symlink /usr/local/bin/node
5 verbose run-script [ 'prestart', 'start', 'poststart' ]
6 info prestart presence-tracker@1.2.0
7 info start presence-tracker@1.2.0
8 verbose unsafe-perm in lifecycle true
9 info presence-tracker@1.2.0 Failed to exec start script
10 verbose stack Error: presence-tracker@1.2.0 start: `babel-node bin/presence`
10 verbose stack spawn ENOENT
10 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/lib/utils/spawn.js:17:16)
10 verbose stack     at ChildProcess.emit (events.js:98:17)
10 verbose stack     at maybeClose (child_process.js:766:16)
10 verbose stack     at Process.ChildProcess._handle.onexit (child_process.js:833:5)
11 verbose pkgid presence-tracker@1.2.0
12 verbose cwd /volumeUSB1/usbshare/Presence
13 error Linux 2.6.32.12
14 error argv "node" "/usr/local/bin/npm" "start"
15 error node v0.10.41
16 error npm  v2.15.1
17 error file sh
18 error code ELIFECYCLE
19 error errno ENOENT
20 error syscall spawn
21 error presence-tracker@1.2.0 start: `babel-node bin/presence`

Any idea on how to continue from here. Obviosly there is something wrong with the babel-node dependency.

MatjaLipu

It looks like it requires binary dependency which is not available for your platform.
I'll remove babel dependency in next release.

Phuturist

MatjaLipu said:

It looks like it requires binary dependency which is not available for your platform.
I'll remove babel dependency in next release.

Ok, great. Looking forward to it playing around with this ...  

MatjaLipu

I've published new version. Presence support app is now installed as npm module (see readme). Use example config file as in readme.
First install presence app and once confirmed working proceed with Homey app.

Until Homey app is approved install it from Github.

Phuturist

Nice one. .. 

Gonna check it out this evening and will report here if I get this working on a Synology NAS. 

gerardz

MatjaLipu said:

I've created an app which supports alternative presence detection.
It supports ARP scan, ping and TCP connect detection.

Unfortunately it is not all that obvious how to configure and run it. It is somewhat installed on my raspberry pi2, but after my initial:
presence-tracker -c /var/local/homey/config.json

It complains after a little while:
listening on 3000
module.js:453
    throw err;
    ^

SyntaxError: /var/local/homey/data.json: Unexpected end of input

data.json? That is nowhere mentioned. I can create the file, but that does not change the throw error. What data.json file does it need where and what is supposed to be in it?

Phuturist

gerardz said:

MatjaLipu said:

I've created an app which supports alternative presence detection.
It supports ARP scan, ping and TCP connect detection.

Unfortunately it is not all that obvious how to configure and run it. It is somewhat installed on my raspberry pi2, but after my initial:
presence-tracker -c /var/local/homey/config.json

It complains after a little while:
listening on 3000
module.js:453
    throw err;
    ^

SyntaxError: /var/local/homey/data.json: Unexpected end of input

data.json? That is nowhere mentioned. I can create the file, but that does not change the throw error. What data.json file does it need where and what is supposed to be in it?

Same kind of issue, when following the readme file it asks me to enter paths to .json files but there are no .json files in the installed npm presence module. There is no data.json anywhere to be found and I could only find a ping.js and tcp-connect.js. Could you elaborate on this @MatjaLipu ?

[EDIT]
When entering the path to the .js files and creating a blank data.json I get the following error when trying to run the server on my Synology NAS. Any clue?

/usr/local/lib/node_modules/presence-tracker/node_modules/koa/lib/application.js:8
const isGeneratorFunction = require('is-generator-function');
^^^^^
SyntaxError: Use of const in strict mode.
    at Module._compile (module.js:439:25)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous> (/usr/local/lib/node_modules/presence-tracker/dist/src/server.js:10:12)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)

[EDIT2] Nevermind, my nodejs version is outdated (0.10.0) and I cant update it because my Synology has a ARMv5 architecture. Looks like this is only going to run on newer Synology NAS models with either ARMv6 and up or Intel x86 architectures.

MatjaLipu

Thank you for your feedback.
These json files are storage files for persisting tracker data.

storage.path = data.json - persist custom device data
ping.path = provider.ping.json - stores configuration about devices we want to monitor (ping)
tcp-connect.path = provider.tcp-connect.json - stores configuration about devices we want to monitor (TCP connect)

All paths can also be absolute (beginning with /).

I've updated package which should work with (updated) default config file.
Please update with   npm install -g presence-tracker

Although you still need node version 4 or greater.

MatjaLipu

One more note on providers. Default config file enables all providers. Just remove providers you don't want to use.
By default only ARP scan auto detects devices. For other providers you need to add devices to tracker.

gerardz

Okay, so, presence-tracker is running on my Raspberry. Requesting something works (http://192.168.1.62:3000/ gives me a list with arp-scanned devices), that URL is configured in the Better Presence settings on the Homey, (under Presence app URL). Yet the tab Providers stays empty, and the Status tab gives me a "Refresh presence"  option that does not change the Status tab.

How can I debug this further? And... should my configured /var/local/homey/data.json still be empty after some interactions with the daemon on port 3000?

And before anyone asks: no, it did not create a systemd script that you can use I chose the old fashioned /etc/init.d/script route to start this daemon.

RobertLijkendijk

I do like what you did creating this app, but I think it's a shame it's only usefull for people with knowledge about Linux... Would be nice if no separate things would be needed, and just installing the app onto homey and set some parameters from the app settings screen would do the trick. Perhaps something can be worked out with the @athom crew to make this happen? I understand the separate presence app installation is needed because of some security things in Homey atm?

PimViergever

{"data":null,"status":0,"config":{"method":"GET","transformRequest":[null],"transformResponse":[null],"url":"http://192.168.1.27:3000/.meta","headers":{"Accept":"application/json, text/plain, */*"}},"statusText":""}

This is the message that appears on top of the 'Presence App Url' page. The rpi mentions that the server is listening on port 3000. I've checked that the config.json is writeable.

After I changed the config.json and changed 'homey.local' to the correct IP-adress, the app is working!

MennoVanGrinsven

I have the app and tracker installed. The tracker is running on a linux server. I have installed arp-scan and added two ping ipv4 adresses. But i do not see anything in the app settings.

I also do not see a data.json being created in the directory i added in the config.json. I see the provider.ping.json and provider.tcp-connect.json, but not the data.json.

MatjaLipu

I've created short setup instructions and configuration documentation.

Hope this answers some questions.

In short:

- app needs CORS headers configured (`allowedOrigins` config)

- to disable provider simply remove it from config
- tracker app needs restart after config change
- current application sandbox model does not allow running su binaries
- I'll add GUI for managing provider devices

willem138

Does this also works for windows?
I got the presence app working on a windows device but for some reason the ARP scan in the app shows nothing but when I run ARP in the command line all devices are present

MatjaLipu

Presence app should work without a problem. I haven't tested arpscan/ping binaries on windows. It looks like ping module have a support for parsing windows binary output. I am not sure about arpscan module.
Can you provide me an output of arpscan and I'll take a look to add windows arpscan support. simply run  and provide me the file.

arp-scan -l > arp-scan.txt

Thanks.

SietseVisser

I am getting a list of MAC addresses in the Settings screen in Homey, but not in the arpscan tab.
Going to http://<ip>/providers/arpscan gives a 'Not Found'.

But the app still works. If you want to test the presence, make sure to use upper case MAC addresses.

The only way for me to reliably test whether a mac address is present or absent is by using the app Better Logic, and then use a Math.js expression. Only fill in the Present label; if you want to capture an Absent event, check the false checkbox in the Math.js expression.

What would be the correct way to use the built-in logic?

Sietse

MennoVanGrinsven

MatjaLipu said:

I've created short setup instructions and configuration documentation.

Hope this answers some questions.

In short:

- app needs CORS headers configured (`allowedOrigins` config)

- to disable provider simply remove it from config
- tracker app needs restart after config change
- current application sandbox model does not allow running su binaries
- I'll add GUI for managing provider devices

Just a question. arp-scan does not auto detect my interface. i need to manually supply that. where can i configure tge arp-scan commandline parameters. 

p0nts

What if going to the url manually (http://192.168.2.5:3000/) provides me with all information of connected devices on the network, but inside the Homey app it does not seem to work after refreshing with the refresh button.

There are no devices seen there, i've already restarted the presense app and also the homey.

willem138

@MatjaLipu  doesn't work..

I think windows doesn't recognize arp-scan only arp..

MatjaLipu

@SietseVisser  Homey app have "Presence changed" trigger which provides you with "Present" (boolean) and "ID" (provider id) tokens. For arpscan provider id is mac address where upper/lower-case depends on arp-scan binary system uses.
Don't understand what is the problem with 'Not found'?

 @MennoVanGrinsven custom args are not yet supported. added to todo.

@p0nts please check if you have updated 'allowedOrigins' in tracker app (this causes cross origin error in dev. tools).

@willem138 arp-scan is not system default binary. Not sure if there is support for arp-scan in windows.

In general, if you have issues showing presence data in Homey app, in Homey Better Presence settings open Chrome developer tools, refresh page and check network and console tabs for any errors.

MarcoWijk

@MatjaLipu not to dis your app. But i don't have a raspberrie, and since Homey isn't configured (yet) to do all of this himself i was wondering if there is another way to get this pressence-function...

@ErikVanDongen I tried to connect to my samsung via your "http://" it showed connection refused. As my samsung has a fixed IP-adress in my network, homey can reach it. Only my samsung refuses acces, as it should. Turning off Wi-Fi on my samsung homey cannot connect or find the IP. Duh....  

Now i had a thought, if connection refused was added as a valid value in a flow, Homey would be able to detect a pressence of a person when his/her phone static ip-adress is defined in a flow or variable. Although connection are refused you would be able to create somekind of boolean trigger. Ip-adress is there but the connection is refused = true, and someone (phone) is home. Ip-adress is not there  = false, so someone (phone) is not at home. 

I know it's a bit of a workaround, but if possible, Homey would be able to detect someone without an account. Perhaps could scan a guestnetwork or defined guest IP's in a DHCP-pool to detect pressences....

Just a thought. Perhaps not even possible.

ericjanvanputten

Okay installed it on a pi 1  borrowed from a friend as a temp and got this far:

- installed raspbian
- installed the presence stuff on pi
- updated allowedorigins
- get info from web url
- installed app on homey
- added url to app in homey
- doesnt do much after  
- i get not found when i 'reload' the app page in homey. 

config.json - that i did with sudo nano config.json

  "presence": {

    "interval": 30000,

    "addedCount": 0,

    "removedCount": 3

  },

  "server": {

    "port": 3000,

    "logRequest": false,

    "allowedOrigins": ["http://192.168.1.81"]

  },

  "storage": {

    "path": "data.json"

  },

  "hooks": [],

  "providers": {

    "arpscan": {},

    "ping": {

      "path": "provider.ping.json"

    },

    "tcp-connect": {

      "path": "provider.tcp-connect.json"

    }

  }

}

and the web URL results of: http://192.168.1.16:3000/

{"20:4E:7F:78:3A:40":{"id":"20:4E:7F:78:3A:40","last":1461876425673,"type":"arpscan","ip":"192.168.1.1","vendor":null,"first":1461876425673},"B4:75:0E:1B:5C:1A":
{"id":"B4:75:0E:1B:5C:1A","last":1461876425674,"type":"arpscan","ip":"192.168.1.2","vendor":null,"first":1461876425674},"B8:27:EB:BC:BE:37":
{"id":"00:17:88:27:8C:A2","last":1461876425676,"type":"arpscan","ip":"192.168.1.12","vendor":"Philips Lighting BV","first":1461876425676},"B4:75:0E:1B:5C:98"

etcetc all kinds of fun stuff 

so what am i doing wrong on the homey app?

MatjaLipu

can you elaborate on "not found in homey"? what url, network tab screenshot.

ericjanvanputten

MatjaLipu said:

can you elaborate on "not found in homey"? what url, network tab screenshot.

I get the following just after selecting the app in settings, if i hit any other tab like status/providers/settings or hit save it disappears.


maybe FYI but im running homey 0.8.32 currently

MatjaLipu

remove the trailing /