this post was submitted on 27 Feb 2025
1 points (100.0% liked)
Privacy
6 readers
2 users here now
Everything about privacy (the confidentiality pillar of security) -- but not restricted to infosec. Offline privacy is also relevant here.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
We're in the process of building our own network location database based on scraping all of the cell tower and Wi-Fi data from Apple's service. Scraping all the cell tower data is quick and will be easy to keep rapidly updated. A contributor scraped more than 2 billion Wi-Fi APs over 3 months.
@GrapheneOS@grapheneos.social Curious about the pros and cons of maintaining a separate DB compared to a public project like BeaconDB...
@downey@floss.social If they provide database downloads in an adequate format usable for our service, we can import it with the Apple data. We're free to use multiple sources of data for our service. We can scrape nearly the entirety of Apple's data and then keep it updated regularly via continuous scraping of the same areas which will avoid much need for anything else though.
This data isn't copyrightable and Apple freely offers it without requiring authentication. It will be the initial basis for our database, but we'll add other sources including an option to send us data from GrapheneOS devices. We'll provide database downloads to support offline network location.
@GrapheneOS@grapheneos.social Very interesting. Thank you!
@GrapheneOS@grapheneos.social All of that is amazing! Great work, I'm exited to see it in action!
I assume for it to work either WiFi of WiFi scanning needs to be enabled though, right?
@jana@social.jsteuernagel.de Yes, either Wi-Fi needs to be enabled or Wi-Fi scanning needs to be enabled to allow doing Wi-Fi scans regardless of the Wi-Fi state. There's a new network location option added in the menu for it. We're going to be adding cell tower support too as a fallback approach it can use if there aren't enough nearby Wi-Fi networks. We've also already essentially fully scraped Apple's cell tower data and it will be much easier to provide that for a fully offline mode than the Wi-Fi AP data.
@GrapheneOS@grapheneos.social
Why not look into ?
@GrapheneOS@grapheneos.social just wondering how this might work? Will this be an internal Android function or via a third party open source app like NeoStumbler?
@x_tof@verkehrswende.social It will be first party GrapheneOS functionality built into the OS similar to our network location client.
@GrapheneOS@grapheneos.social Why are you using the Apple's service and not Google's for example? I thought it was better. Is this because of legal issues?
@astroboy@fosstodon.org Apple's service is better for our purposes because it provides the Wi-Fi AP and cell tower location data for the area which can be locally cached in memory. It also means we had to implement a local algorithm for determining the location based on it. Apple's API enables us to scrape their service for data to build our own database by crawling through the data. Google's service would currently work better because the algorithm is smarter than ours but we can improve it to work as well.
@astroboy@fosstodon.org Since we have a local algorithm already, we've already a lot of the way towards implementing entirely offline network location support. We just need to provide a database based on scraping Apple's data for local queries of networks and networks near those. We already have the code to calculate the position from it locally. Google's service does the calculation on their service which prevents mass scraping the data in a similar way and also prevents short term offline use via a cache.
@astroboy @GrapheneOS Google’s is alreadY available via Play Services
@BucciaBuccia @GrapheneOS True, but it's not available offline, and not through GrapheneOS's proxy. Or maybe I'm wrong about the latter, I actually don't know.
@astroboy @BucciaBuccia We can't scrape Google's data due to how the service is designed and also can't have a similar in-memory cache system. Apple's service is better for our usage. We could add Google's service as an option but it would be entirely server-based with no way to support offline usage either short term via our in-memory cache approach or fully offline via database downloads. Our focus is working towards fully offline network location support so we aren't going to bother with it.
@GrapheneOS@grapheneos.social @BucciaBuccia@mastodon.social Thank you for clarifying. I strongly agree location data should be offline in the end.
@GrapheneOS@grapheneos.social
is this going to be worldwide?
@davidwebb@privacysafe.social Yes. We used Mozilla's cell tower database as an initial seed to scrape essentially all of Apple's cell tower data. We'll be scraping all of their Wi-Fi AP data from around the world too. We can also add other sources of data but we don't really need it beyond having a path forward if Apple ever goes through a multiple year process to phase out having a publicly available network location service. It would be hard for them to do that since their existing devices depend on it.
@GrapheneOS@grapheneos.social Thank you for clarifying.