What is your weirdest self-hosting problem you've had to solve? For me: Mouse inside the server.

[gekkonen@sopuli.xyz]

The mouse did not survive interrogation.

Also mouse not pictured.

[ebby@lemmy.ssba.com]

I lost a raspberry pi for a couple months. Just couldn't find the darn thing. It was working fine serving away.

Turns out it was in the basement. I forgot I moved it to be closer to my tilt Hydrometer.

[gekkonen@sopuli.xyz]

To be fair, those things are really tiny!

[darkassassin07@lemmy.ca]

For a while, the windows machine I was using kept randomly becoming unresponsive (both via network, as well as the keyboard+mouse) and I just could not figure out why. Everytime it did, I'd have to force a shutdown by holding the power button until it shutoff, then press it again to start back up. Sometimes this happened while I wasn't home, so it had to just stay offline until I was.

I got sick of having to do this; so I replaced the power button with a transistor and a RPI. The PI would ping the server every 5min; if it failed to get a response 3 times in a row, it would trigger the transistor for 10sec, release for 3sec, then press and release again; forcing a poweroff then starting the machine again. It'd also write these events to a log file. I think it was around twice a week ish.

Never did figure out what the cause of the lockup was; but it stopped when I replaced windows with Debian.

[devaly@ani.social]

Had a problem where my ports would suddenly close.
At first I thought it was Telekom, then the modem, then the NAS.

After much investigation I discovered that restarting my router would solve it.

After some more weeks I discovered that the crawling bots were triggering some stateful tplink firewall that would close everything until rebooted.

So the router in the middle of the setup was the problem, not the edges.

[uxyivrljperl@lemmy.world]

Well at home i only have problems with spiders in my racks, not like the racks i have seen at work.
Highlights are toiletpaper, chipmunks+nuts and worst in a school server rack a gun.

[shimitar@downonthestreet.eu]

I use e self made wood cabinet in the attic where also the PV inverter is located. I used Velcro to strip on the wall (brick wall) stuff like switch firewall etc, only to find those collapsed on the server itself every time. No matter which glue or adhesive i used.

At the end, a strong unbranded Chinese double sided tape was better and still strong after all big western brand failed over time.

Maybe a mix of heat and uneven surface was the culprit, who knows.

[rimu@piefed.social]

Nice chonky heatsink, goddam

[uxyivrljperl@lemmy.world]

Had a zyxel nas that got unresponsive after 10-15days of uptime, thankfully it hat a scheduled shutdown/reboot setting. Runs since 2014 with a reboot every Mondy 3am.

[gekkonen@sopuli.xyz]

I used this, among other things, for my personal Blender renderfarm. Cooling was a must.

[blurry@feddit.org]

Cat vomit... my cat hit my raspberry pi.

[rimu@piefed.social]

haha looks like you're aaalmost at the point where water cooling is compulsory.

Is it a Threadripper? I built a couple of those for architectural rendering, such absolute beasts.

[nnullzz@lemmy.world]

One day, our internet just stopped working entirely. Our ISP was fine, router good, but no outside connection and I couldn’t access local devices. I have two Unraid servers. I took a quick glance at them, they were powered on, all looked ok. I decide to power cycle everything. Still no internet. Then I realize that I can still access Google from my phone when on WiFi.. hmm. That would mean… an issue with pi-hole!

I begin to try to diagnose from my phone with a hotspot to my computer. I’m getting warmer but some things just aren’t making sense. I go back and stare at the server running pi-hole. Then it fucking hits me.

I have the pretty odd and totally not ideal home network. I have plans to upgrade everything but currently I have two wifi networks tied to the same WAN. One network is run off of access points so I can get Ethernet to odd areas of the property. The Unraid server hosting pi-hole was connected to an AP and the Ethernet cable I used didn’t have a clip, so it just barely unseated itself from it, breaking the connection. The router+pi-hole config meant also breaking the internet if anything happened to pi-hole or that server. I swapped it for a better cable and it all instantly started working.

Spent that weekend sorting out the network so it doesn’t happen again. It was definitely a head-scratcher.

[rimu@piefed.social]

It's always dns at least half the time

[tofu@lemmy.nocturnal.garden]

Pihole is cool, but fucked me over as well more than once (not the pihole itself but the host). It's on its own hardware now, but I plan to add a second one to reduce the single point of failure.

[gekkonen@sopuli.xyz]

I honestly don't remember now as this was over 10 years ago.

[nnullzz@lemmy.world]

Haven’t thought of that tbh. Would they both just share configuration? Curious how leasing would be split/shared between them. Off to research!

[tenchiken@anarchist.nexus]

Mine would be when I had a 48 bay disk array / JBOD fail on me... badly. After a storm, it killed the larger card that allowed for me to get many of the drives into a PCIe 16x slot, and I was relegated down to only getting ~8 disks per box made from spare hardware. A single box I got 16 going. Add to this mix an SSD for mid-line caching.

These were all running bcache on top of mdraid... One single mount.

Yes I understand how obnoxiously stupid it was to run RAID6 on a 48 disk volume. It was almost all just stuff I could re-acquire over time, not irreplaceable things.

I just HAD to solve this one though.

In come several spare chassis / mobo etc.... get a bunch of drives powered and on /dev/ , move to the next.

A couple spare gigabit switches...

several gigabit NICs...

two explicit paths for each machine...

a bit of iSCSI magic, and one machine now had the physical disks all exposed to it... mdadm --assemble blah blah, bit of UUID chaos...

It's surprising that while a bit speed limited (I think I got just around 110MB/sec reads), it was nicely performant for what a huge mess of wires and disks just strewn out around my rack.

Managed to evacuate all I needed without much issue once I got that going. Now, I try to keep my arrays under 16 drives at a time, or keep a very rigid policy of "I can lose this and don't care" vs "this box gets RAID10 and/or offsite backups nightly".

back up your critical stuff people!

[tofu@lemmy.nocturnal.garden]

There's some sync mode but I haven't really looked into it yet.

[nnullzz@lemmy.world]

Makes sense.