Request For Comment - Solder API v0.8

[Indemnity83]

Documentation: http://docs.technicsolder.apiary.io
Implementation: indemnity83/technicsolder
Issues: indemnity83/technicsolder#issues
Proposals: indemnity83/technicsolder#pull-requests

Abstract

The current API has worked really well for its purpose, which is to have users create client modpacks in a web browser, and let the launcher know what needs to be pulled together to deploy that configuration.

However, it is NOT very good at allowing third party tools to facilitate the process for a user. The endpoints are all read only, and their return datasets are inconsistant from endpoint to endpoint.

I propose developing a compete JSON API v1.0 implementation of the TechnicSolder platform. In order to modernize the API, I've also re-built the core of the application using Laravel 5.3 and that implementation too is up for comment. I have done a lot of conceptual, and early implementation work toward that goal and would like to get feedback (particuarly from @GenPage) before going on any further.

Backward Compatability

Existing api endpoints use singular nouns (mod, modpack) and the proposed v0.8 endpoints MUST use plural nouns (mods, modpacks, builds, etc.), which will ensure FULL BACKWARD COMPATABILITY. This is critical to the success of the update, as the Technic system is widly distributed and there are many components that rely on the v0.7 API.

The v0.7 API would be deprecated, but would likely be maintained until at least the TechnicLauncher and TechnicPack.net consumers were updated, and significantly adopted.

API Documentation

The living API Documentation can be found at http://docs.technicsolder.apiary.io, the raw version of the document is pulled from the apiary.apib file in the root of the development branch.

Contributing

Please make specific comments/suggestions as issues in the forked repository. Similarly if you'd like to suggest a change to the documentation please submit it as a pull request against the forked repository.

Notes on Proof of Concept Stability

The Apiary API documentation, as well as the Application impementation are ovbiously subject to significant change as this proposal is discussed, and if/until they are pulled into the master of the technicpack/technicsolder repository they should be considered unstable, and subject to breaking changes (for example, changes to the database structure WILL be made to the existing migrations files).

Breaking changes will be noted where known in commit messages, but no garentee is made and the application in its current form should not be considered fit for production use.

[GenPage]

Hey @Indemnity83!

Thank you for the contribution here and I'm sorry I haven't gotten back sooner. My first thoughts is that it touches on the main concerns I have while moving forward,

The main holdup for the Technic team is the lack of a launcher developer at this time to implement the new API changes. I'm glad to see backwards compatibility talked and addressed as that would be the main concern/block.

I will be reviewing this more deeply in the coming weeks. My main focus/time has been addressing the TechnicPack move to DO infra, and subsequent changes to improve performance and reliability for the entire platform.

[Indemnity83]

No problem @GenPage; I realize this is a lot to digest.

I'm excited to push some significant progress into the project, but at the same time I'm only one person working in a vacuum. So take some time, see how this fits in the road-map of the TechnicPack team as well as the desires of the community as a whole.

[sct]

Wow @Indemnity83 first let me say thank you for taking the time to put together something like this. When Solder was created it was always supposed to be a community project of sorts but we really never got any sort of contribution on this level. So thank you for spending your valuable time to help Solder become.. whatever's next!

Your Apiary docs are interesting, because a year ago we started laying the groundwork for what would be the new Technic Solder. Unfortunately our plans came to a screeching halt when we lost our primary launcher dev. Our plans for the new Solder would require a significant launcher overhaul and it just wasn't possible at the time to continue development on the private, experimental Solder branch. You can see some of our own work at this Apiary. http://docs.solder.apiary.io/

You may or may not have seen that yet. If you have you may notice some glaring similarities in the design between your concept and ours. I do like that you are following the JSON spec and to be clear our API blueprint was very rough and in early stages. Although I did technically have a somewhat functioning version of the API running in Node.

We have always wanted to abstract Solder's API out another level. That was the plan of the new API. We wanted to handle mods as "resources" instead of just always being mods. For example I am pretty sure in our docs we even treat mods as just "files." You specify the resource as the file type, and in the options parameter you can assign the directory it gets dropped. And of course, the classic "zip" type which just extracts over the base directory.

Unfortunately this type of implementation can't really go anywhere without the launcher being rewritten. But this is why I like your idea! We can begin to work towards this goal while still maintaining backwards compatibility with 0.7. As you mentioned in #356, a worker that could package up mods into the classic zip form for compatibility would be great in this sort of situation. Then, when we can eventually progress launcher development, we can work towards full 0.8+ support.

I really like where you are going with this and like @GenPage said I will have to take some time to actually play with your fork and dig into it a little bit. As of now I have only looked through your API docs. I like what I am seeing though.

Let's keep in touch and keep pushing this forward. If you have any questions or ideas feel free to run them by Genpage or me in Discord (if you are on there) or just in here.

[Indemnity83]

@sct I had not known about the API you had going on apiary already; it is funny how similar we ended up 👍

I do use Discord (although not often). If you want to discuss there feel free to send me a message (@Indemnity83 there too); I added you as a fried so we could so PMs for some of this, might be faster.

I totally hear you on the lack of a launcher developer being an issue; but I don't think it should slow down the development of an API; ultimately the API is the common ground that both development teams work toward so these are good discussions to have.

Looking through your API blueprint vs Mine there are a few minor differences that are mostly semantics and a couple more significant architecture decisions we should chat about.

Semantics

Mods vs Resources

I concede that mod isn't appropriate for the direction we're all headed in, but resource might be confusing with Minecraft Resource Packs. I'll update my document to resource for now though.

Release vs Versions

I had modeled this as release just because I felt version was somewhat ambiguous, but since I can't make any better argument than that, again I'll update my documentation and implementation back to version.

Architecture Differences

Resource Versions have ONE file vs Resource Versions have MULTIPLE files

This one is more significant; your spec has a single file per resource version, and that resource version has a type as well as an ability to set a path for the file on the client end. In my implementation proposal in #356 I assumed that a resource version could have multiple files (assets), each with their own location attribute (again semantics, path vs location). My thoughts were a mod and its related config file in a single "package".

The single file per resource version certainly simplifies things; but I don't think it would scale well. I could be convinced otherwise though.

Self Defined API vs JSON API Standard

Not much to say here; just need to choose a direction. the JSON API is a little more work to implement but is very well thought out and allows developers for clients to use pre built tools to consume the API instead of having to write from scratch each time. My recommendation is to use the standard solution so we can focus on the application, not how the JSON is formatted.

Password token Auth vs OAuth2

Similar to above; choice is between coming up with something and making sure its well defined, or implementing a standard. Again, I recommend sticking with standards based solutions to common problems.

Not addressed

Users endpoints

You have included some user management endpoints in your API, I left them out of mine assuming that user management would be done through the web interface. They're easily added though so I can do that for completeness.

[Indemnity83]

I spent some time this afternoon and evening doing an opinionated merge of the two API specs; I also put a little work into making the document a little more prescriptive in areas since when its done, it should in fact be a specification.

[sct]

Phew okay a lot to talk about here huh. Okay so for the similarities with "Resource Packs" I think people will just need to deal with that 😄

Our Resource object took only file, you're right. I like your idea better. No reason it can't do multiple files.

I also agree on the JSON Standard.

For Auth, it's been awhile so I don't remember all the details exactly but I am pretty sure we were using JWT standard for Auth. OAuth2 seemed like it was a little overkill at the time during design. Either way it doesn't matter. They both accomplish the same thing I am okay with OAuth2.

We had a User endpoint because the new Solder was going to be JUST the API. The frontend being basically just an interface to control everything. The API was standalone. Then the frontend was written in React to plug into the API. But auth, users, modpacks, resources and anything else Solder may have come to do would have been done through API calls. Which would allow anyone to build their own tool for Solder in any language they wish.

Either way haven't taken a look at your merge yet. Let me do that now 👍

[Indemnity83]

For Auth, it's been awhile so I don't remember all the details exactly but I am pretty sure we were using JWT standard for Auth. OAuth2 seemed like it was a little overkill at the time during design. Either way it doesn't matter. They both accomplish the same thing I am okay with OAuth2.

Perfect, the Laravel Passport component I pulled in for Auth supports JWT too.