I didn’t explain what I meant very well. To scrape a website you don’t need to understand robots.txt, implementing robots.txt is something you do to be a good netizen. But to get like info from Lemmy, implementing ActivityPub is a requirement.
Now I’ll admit, it’s not a great system and I do wish we had something better, but I also don’t think “this isn’t a good way to communicate preferences” is a good reason to ignore them.
idk, the label is also an honor system, if it can be just ignored like robots.txt.
I didn’t explain what I meant very well. To scrape a website you don’t need to understand robots.txt, implementing robots.txt is something you do to be a good netizen. But to get like info from Lemmy, implementing ActivityPub is a requirement.
Now I’ll admit, it’s not a great system and I do wish we had something better, but I also don’t think “this isn’t a good way to communicate preferences” is a good reason to ignore them.