So I worry that there will be a ban on adult sites where I live evntually, and I want to have a backup of e6 just in case it gets banned or blocked. I have a script (re621) where I can mass download images including tags and meta data. But I want to be able to have a way to locally view them, maybe booru style or a clone of e6. Does anyone know of a program or e6 clone that can run local and pull from an assigned static folder? It would just be nice to have a tag search function beyond scrolling through my files manually. Im slightly technically inclined, I can learn and figure stuff out, but im not super knowledgeable on like web dev and application hosting and stuff. Its totally fine if it just runs locally on my desktop, but bonus if i can toss it in a docker container and pull from a NAS or something. But didn't know if anyone had a method to recommend. Thanks!
Topic: e621 backup local hosting question
Posted under General
The site is open source; you can click the version number on the bottom of every page but the home page to go to the GitHub repository. If you want a less demanding solution, I've heard good things about Hydrus, & there's also this relatively new home-grown solution.
If you really want a full-as-possible backup, I'd look at our DB Exports. See our API help page for more info.
aacafah said:
The site is open source; you can click the version number on the bottom of every page but the home page to go to the GitHub repository. If you want a less demanding solution, I've heard good things about Hydrus, & there's also this relatively new home-grown solution.If you really want a full-as-possible backup, I'd look at our DB Exports. See our API help page for more info.
thank you for all that info thats super helpful! I tried to use Hydrus a couple years ago and had a lot of issues with it. certain posts wouldnt show when i search tags, even though the tags were good on the posts, and the file sorting is a nightmare when i had to pull everything out. I like the idea of hydrus, and maybe its fixed now, but then it was pretty buggy to use. but ill look into the other stuff and the github, thanks so much!
aacafah said:
The site is open source; you can click the version number on the bottom of every page but the home page to go to the GitHub repository.
Just checked this, and this is literally perfect, Im currently working on a new home server to run jellyfin and being able to chuck this in a docker should work great! thank you!
I would definitely recommend Hydrus.
It gets updates about every week and the developer regularly answers questions from users if you have trouble.
Hydrus has changed a lot. I remember I tried it a year or two ago and I couldn't make sense of it at all. I got back into it recently after getting tired of Imgbrd-Grabber + Digikam (both great software, just not meeting my needs very well) and Hydrus was much easier to set up and use than I remembered.
johannfaust said:
So I worry that there will be a ban on adult sites where I live evntually
Another nice thing to have in this circumstance is a VPN, I recommend Mullvad specifically for this use case, that way you can still browse and save stuff.
I have a shitty script that downloads files and sort them into folders lol.
Some time ago I saw someone recommend szurubooru plus a browser extension that scrapes whatever boou page you're on and uploads to it. It didn't support e621 unfortunately but if you also use other boorus you might want to check it out.
banhday said:
I have a shitty script that downloads files and sort them into folders lol.
Some time ago I saw someone recommend szurubooru plus a browser extension that scrapes whatever boou page you're on and uploads to it. It didn't support e621 unfortunately but if you also use other boorus you might want to check it out.
Chances are if it supports danbooru, it can be massaged into supporting e6 pretty easily (since e6 is built from a danbooru base)
Szuru's browser extension is supposed to support e621 based on the documentation I read, but seems to be broken. It works on vanilla Danbooru and has a fallback mode that works for most things, but lacks tag support.
Szurubooru itself is simpler and easier to manage, and a LOT easier to set up and maintain, but lacks a lot of the features that e621 has in turn. (Granted I am not particularly apt for programming, so take my experience with a tablespoon of salt.)
I would like to eventually spin up a local instance of e621ng and modify it, or create some kind of middleware, that can serve to turn it into a local mirror. Essentially a BMR/BDR for any part of e621 that you have shown interest in. (In IT, we have a type of Backup and Disaster Recovery, called Bare Metal Recovery, which is a clone of a server stored on a backup device. If the production server goes down, you can temporarily spin up the backup on the backup device, while you work to get the main server back online.)
I currently use McSib's e621_downloader which has served well for years, but seems abandoned and the increasing instance of random, inexplicable errors makes me wonder if it's on its last leg. Plus, like Johann mentioned, a folder with a loosely sorted pile of untagged images is not a great experience beyond a bare minimum backup. I serial favorite and hoard neat art like a dragon hoards gold - in much more excess than I would ever know what to do with - so that 'scales' poorly for me.
The idea is your favorites, plus any artist, pool, post, you add to a list; their tags, the tags' wikis/categories, comments, maybe the commenters' avatar posts (minus anything you've blacklisted) - can be copied into the local instance, so that you effectively have an entire local clone of "the site" from your point of view. You still use the proper site day to day to find new art, etc., but if a disaster arises, you have a "last good state" to run locally.
Where I am, I don't think they would ever actually ban adult sites. Their strategy instead is to harass people who use those sites by making it impractical to reach them without having to jump through hoops and give away any sense of privacy. Some sites refuse to connect because of that, others want you to hand out your ID/face to some random company that conveniently bypasses the 4th amendment by being a private entity doing the gov's work for it, some pretend like nothing has changed and just work normally. Everyone just uses a VPN, so then next they just try banning VPNs, etc. etc.