Austin Elixir

Alembic

2016-06-20

Author

	limhoff@csd.org	Kronic.Deth@gmail.com
	@KronicDeth	@KronicDeth
		@KronicDeth

I am a Senior Software Engineer at Communication Service for the Deaf (CSD). CSD is fully remote for the development team and most of the non-profit outside the call centers. I work on Vineya, a market place and SaaS to allow the Deaf community to select their own American Sign Language (ASL) interpreter. The current, market-place-only version of Vineya is a Ruby application that was developed before I joined CSD. The SaaS version, under development, is a mix of Ruby applications and Elixir applications.

In my free time I am the maintainer of IntelliJ Elixir, an Elixir plugin for JetBrains IDEs, like IntellIJ IDEA and Rubymine.

API

Step	JaSerializer	Your Code	Alembic
1	`ContentTypeNegotiation`
2	`Deserializer`
3		action
4			`Document.from_json`
5			`Document.to_params`
6			`ToParams.nested_to_foreign_keys`
7			`Fetch.from_params`
8			`Fetch.Includes.to_preloads`
9	`PhoenixView`

When I announced the release of Alembic, there were multiple people wondering how it compared to JaSerializer, but for Vineya, we use JaSerializer with Alembic. We use JaSerializer's plugs to deal with the content-type and accept headers and to convert from JSONAPI's hyphenated keys to the underscored keys used by Ecto. Once inside each action, we use Alembic to (4) validate that the params are valid JSONAPI document; (5 & 6) convert to the params format used by Ecto.Changeset; and (8) to convert included relationships to association preloads before using JaSerializer again to render the view.

When I started at CSD in November 2015, JaSerializer did not have Params.to_attributes, which was only added in April 2016, so params arrived at the Phoenix.Controller action, but there was no validation that the params were even structured as a proper JSONAPI document. We could manually pick out the data attributes and relationships that were needed for each controller and action, but that didn't seem right to me. My understanding of the purpose of JSONAPI was to make the encoding and decoding of resources more regular, so the code for any action across all controllers for JSONAPI resources should be the same. Furthermore, the JSONAPI spec outlines a format for errors and when they should be returned, so I knew that the error handling should be extractable to its own library, so it could get out of the business logic of each action.

`Document.from_json`

Empty Single Resource

Present Single Resource

Present Single Resource Identifier

Present Collection

Relationships and Includes

Errors

Ok, that's great, Alembic can parse JSON into structs, but that doesn't buy us much. What we want is for Alembic to automatically handle error reporting, so if the json is not a valid JSONAPI document, instead of returning {:ok, Document.t}, Alembic will return {:error, Document.t}

If there's an error, how can Alembic return a Document still? It's not a partial Document containing only the valid parts of the JSON, but instead a JSONAPI error document that can be re-encoded and sent back to the sender.

The format of detail and meta isn't that specific in the JSONAPI spec, but in general Alembic tries to include machine parsable presentations of anything from details in meta, so that you can construct your own error messages if you don't like the detail message.

Nested Errors

FromJson.from_json

`Error.descend`

`FromJson.merge`

`FromJson.reduce`

So, how do we use this nice error handling in our controllers? After JaSerializer.Deserializer has converted the hyphenated params to underscores, the action gets the params, and uses those as the json for Document.from_json.

In our InterpreterServer app for Vineya, I've made a module, InterpreterServer.Controller.Ecto, that has common actions for each of the REST actions required by JSONAPI, so for the create action, it calls IntepreterServer.Controller.Ecto.create.

The general structure of the controller functions uses a happy path with in action function and then the sad path is handled by each method being called generating the error response. This effectively works like halting the Plug.Conn, but I don't have to setup guard clauses at the top of the controller to have plugs used for only one action.

valid_create_changeset and preloads_from_include_param both use Alembic, so we'll focus on those functions.

A JSONAPI Document is at most two levels deep: The Resource(s) in data and then any direct or indirect relationships are grouped together in included, but the nested params format has no fixed depth, so to_params needs to pass a lookup table keyed by type and id.

However, the lookup table wasn't enough because the JSONAPI allows ResourceIdentifiers, or effectively pointers, AND it represents has_one and belongs_to associations the same way: as to-one relationships, so when both the Resource with the belongs_to association and the related resource with the has_one are included in a JSONAPI document, you get circular references in the ResourceIdentifiers in the relationships. converted_by_id_by_type prevents the circular references from leading to infinite recursion by tracking the {type, id} pairs that have already been converted from JSONAPI to nested params.

API

Step	JaSerializer	Your Code	Alembic
1	`ContentTypeNegotiation`
2	`Deserializer`
3		action
4			`Document.from_json`
5			`Document.to_params`
6			`ToParams.nested_to_foreign_keys`
7			`Fetch.from_params`
8			`Fetch.Includes.to_preloads`
9	`PhoenixView`

Client

JSONAPI	Ecto.Schema
attributes	fields
relationships	associations

This may seem like I'm overly tying the relationships to the associations, but none of this code cares that an Ecto.Repo exists. The Ecto.Schema modules are only used as a convenient way to declare the casting rules for the attributes and relationships, mapping JSONAPI Resource attributes to Ecto.Schema fields and relationships to associations. The Ecto.Schema modules don't have to be backed by a database table and aren't for our usage of RPC clients.

It so happens that for our API controllers, the Ecto.Schema module used to cast the JSONAPI to structs is the same as the database table, but that doesn't have to be the case and is only that way because none of our JSONAPI resources are composite views of database tables.

Document.to_ecto_schema eventually calls ToEctoSchema.to_ecto_schema, which uses Ecto.Changeset.cast to apply the casting rules defined by the Ecto.Schema module fields. It does not use changeset, because for the use case of a Client, there is no validation of the attribute and associations because whatever the Server sent is the truth.

Because to_ecto_schema uses only cast and __schema__, which are available to any module using Ecto.Schema's schema DSL, there's no new DSLs needed to use an Ecto.Schema module for to_ecto_schema.

The to_string in put_named_association is the code that cements that relationship names (after prior reformatting from hyphenated to underscored) must match association names. If one wanted to make them different, the to_string could be changed to a transformation function that is passed in. PRs welcome.

	Client	Server
API	✓	✓
RPC	✓	✓