Dawarich 1.14.1 - now rendering millions of points in seconds!

[frey@lemmy.world]

Hi there my dear people of r/selfhosted! Fresh bunch of Dawarich news is due, make yourself comfortable :)
To whom it may concern, Dawarich — is your (is and always will be) a free open source self-hostable alternative to Google Timeline.

AI Disclosure (as per https://lemmy.world/post/49320548):

Implementation - Generated

Testing - Generated

Design - Generated

---

Our links:

Github: https://github.com/Freika/dawarich

Website: https://dawarich.app/

iOS app: https://apps.apple.com/us/app/dawarich/id6739544999

Android app: https://play.google.com/store/apps/details?id=app.dawarich.Dawarich

---

Nine releases, what a month! And not a puny ones, this time we managed to speedrocket Dawarich performance in browser.
If you have hundreds of thousands (or even millions!) of points and you'd love to see them all on the map, previously that would be not an easy task for Dawarich. It would crash your browser, as many times as you try loading the data. But no more! Our very own community member, the man behind the Docker image we're using to run Photon, Robin (https://github.com/rtuszik) suggested a fresh approach to loading points: just render them as map tiles! Together we brought this idea to life and now my own dataset consisting of 957,350 geo points gets loaded in less than 1 second! That's kinda insane to me, to be honest. And yeah, instead of ~590MB of JSON transferred to your browser, it took uhhh 150KB? Just don't forget to enable the toggle in the map settings.

---

If you want to drag your points, do disable it though, seamless points editing will be added bit later. Oh, and tracks are being now served in the same manner, too. And routes, and fog of war layers. It's all a lot faster now!
Dawarich now speaks 6 languages! German, Spanish, French, Polish and Catalan were added to the default English. Simplified Chinese is on the way, and feel free to suggest your own locales :) You can change it in Settings -> General

---

Visit detection was rebuilt from scratch and it's finally useful. God knows I procrastinated this feature long enough, but now it's in a very good shape. Suggests real places I visited, and community members are also confirming it's not just me — for them it also yielded very relevant results. By the way, you'll no longer see the annoying "99+" counter of the visits on your timeline button. We all hated the feeling of an unfinished job but let's be real ain't nobody got time to click all the suggested visits. Now they are just there, without the irritating counter, sitting patiently, waiting for you to delete them or... just let them be. They are precise enough now, after all, so why not, right?

Deleting a visit also leaves an invisible tombstone now, so the visit won't be resuggested again. And the Timeline panel got much cleaner. I'm still not entirely satisfied with its state, so some changes are due, I want smoother UX.

By the way, one of former Dawarich user mentioned something like we're making the Timeline "less and less useful with each release", so here I'd like to use the opportunity and ask you guys, do you share same feeling? What's good about Timeline, what's not so good? I'd love it to be as useful and convenient as possible, so your feedback is very important. Thank you!

Transportation modes, also rebuilt. This one is a lot trickier and even Google itself fails to properly attribute the way a human being moved (suggesting driving over train ride for example), but it's still got significantly more reliable. I have some cool work ahead of me to make it even better, but that's a secret.

---

Videos are finally here! God I love videos. The Video Studio first was released as a free tool on our website (https://dawarich.app/tools/route-video-maker/), and then I added it to Dawarich itself, which makes it easier to use your own routes to create a video.

Check it out!

And here's an actual video produced by Dawarich Video Studio:

Also, the "Share" button on the map page now also has a way for you to create and share a poster or a video. Your memories were never more material!

Rendering is being done in your browser, and it allowed me to not cram another backend service into the Dawarich stack, neat.

---

Map v1 (Leaflet) is gone. I'm surprised nobody really complained about it. I'll take it as a sign that Maplibre version is a lot better!

---

Geocoding settings are now available in Settings->Integrations, with selection of providers supported by Dawarich and appropriate settings available for them. The ENV vars are still there and take precedence over what's configured in the UI, so no breaking changes.

---

Imports got smarter:

• Google Takeout & Polarsteps now extract visits, named places, tracks and the source app's tication — not just raw GPS points. Google's own place names (Home, Work…) are kept. Older imports: "Extract additional data" in the Imports list. "Remove extracted data" undoes it without touching your points.
• FIT files with developer data fields (Wahoo devices) now import instead of failing to parse
• Traccar KML exports now import LineString points instead of finishing with zero.
• Google Timeline: fixed altitude/accuracy/speed being silently dropped from rawSignals, and estamps no longer scramble path order.
• Imports finishing with zero saved points now tell you instead of completing silently.
• Tracks no longer stitch two different devices together (Records.json contains every device

---

Tracks calculation got a lot smarter: fewer skims along the route, better anomaly points detection, better distinction between different devices tracking simultaneously, hence stats distance calculation now has better precision.

---

Preparation work is being done for the Family feature coming to our mobile apps. We're currently testing it internally and are hoping to release it there soon. Also, OwnTracks' HTTP mode now can show your family members in the app: every upload is answered witveryone sharing with you.

---

Database optimisation works are also in progress: most of the August releases are containing some changes to how Dawarich writes and serves your data, it may lead to bigger space consumption. In the end, when the process is done (hopefully before end of September), I'll provide an instruction on how to shrink your databases with no data loss. If you'd like to do it now — do the full vacuum to the points table. For the rest of you — please wait for the next post or keep an eye on our changelogs, it will be there.

⚠️ If you're upgrading from 1.10.x, do these once

• Restart the app after migrations have run — until then, deleting a visit looks like it worked without saving.
• Settings -> Visits -> re-detect (self-hosted does this fleet-wide automatically; SKIP_VISITS_FLEET_REDETECT=1 to opt out).
• Map settings -> Transportation Mode Detection → "Re-classify my history."
• Map v2 -> Settings -> "Recalculate tracks & stats" to apply the gap-distance fix to past months.
• Expect background backfills to run for a while on big histories. Progress is under Background jobs settings.

---

Mobile app UI was updated! It now includes better navigation between days, you can select date range to show your data, and you can also replay your day on the map. We added new layers: points and heatmap, more to follow.

If you're hosting Dawarich behind cloudflared or similar gateway, you can now benefit from custom HTTP headers to connect your mobile app to your Dawarich instance.

We added Smart Tracking mode for Android and Continuous Tracking mode for iOS. Enable precise tracking mode to see the toggle. Both modes use motion-aware updates to toggle tracking on and off to save your battery life.

---

Well, I guess, that's it for today?

And as always, our links one more time to save you a scroll:

Github: https://github.com/Freika/dawarich

Website: https://dawarich.app/

iOS app: https://apps.apple.com/us/app/dawarich/id6739544999

Android app: https://play.google.com/store/apps/details?id=app.dawarich.Dawarich

Donate: https://www.patreon.com/freika / https://github.com/sponsors/Freika/

---

P.S. 1st of October I'll be speaking at Berlin's Geomob event ( https://thegeomob.com/ ) come and say hi!

See y'all soon!