Each test starts up a short-lived & completely hermetic environment. Its own database, its own S2S+C2S (federating & social) set of actors, so multiple folks can use the automated tests without interfering with one another.
Still a lot more actual tests that need to be written. Contributions welcome!
Ugly screenshots lol
It is up to 22 tests, been starting on the S2S stuff. While some instructions await you to input something to fetch, many of these S2S tests now give you instructions of sending an actor an activity, so it can continue the testing process automatically upon receipt.
This, at a practical level, means I had to add webfinger as an optional technology to support during a test run. Which means instructions will list both the IRI and the webfinger for the generated test actor(s) for that test run.
Alright, I created an issue for interested parties to follow for the unofficial #activitypub test suite implementation:
It lists the mapping of the official test suite's tests over to the ones I'm hammering out using #gofed.
Unfortunately there's not a good way to track this, I don't think edits to the OP trigger notifications so I'll have to find a balance between editing OP and pinging the thread. Boo github (I don't have the cycles to migrate & deal with #golang consequences).
I am halfway towards completing the "Common" and "S2S" tests in the official test suite, as automated tests for the unofficial suite. There's now 44 such unofficial tests, which map to 23 of the 46 original ones. Unfortunately, one test still relies on the honor of the test-user.
It's not as "automated" as I like, so I think I need a new word. The server definitely sends/receives and diagnoses the software under test, but most tests require additional input or actions to be done by the test-user.
Whew! All the Common and S2S MUST cases are covered in the testsuite.
@icedquinn @lanodan One part of me wants to be silly and say "The internet is a sphere, there are no corners!" or "yeah but now you can write assembly for your backends *and* frontends" but also I had never heard of seaside before so in all seriousness thanks for mentioning it -- learning about this is great!
@cj could you please put a normal font on the site? this pixelated one is completely unreadable.
@felix Yep! Do you have a suggested monospace one?
@cj sorry i'm the wrong person to ask about fonts.
also it seems like the test randomly fails and doesnt do anything, like here (or might be because i didnt check the webfinger box).
@cj looks like fetching an actor from lemmy fails because we only look for the header `Accept: application/activity+json`, if its anything different (like yours with the profile link), it will use the html route instead.
@felix OK, font is updated
@cj ah, our priority is federation between lemmy instances, so actual compliance with the spec is still going to take a while. if the test is specifically about spec compliance, then its not for us at this point.
@felix Gotcha. Yeah right now the testsuite only knows about how to test [a subset of] the ActivityPub spec. It doesn't have Mastodon, Pleroma, lemmy, nor any other kind of app-specific tests for compatibility.
But it could one day.
This Mastodon instance is for people interested in technology. Discussions aren't limited to technology, because tech folks shouldn't be limited to technology either!