Custom and Enum Data Types

Please watch the videos under Resources before class.

In your Computer Science education you have learned that even for Booleans, which can be expressed in a single bit, all values in a computer (including a Postgres Database) take up at least 1 full Byte (8 bits). Recall and discuss with your partner why that is the case.

The Goal

This brings us to todays challenge. For whatever reason we will have a resource (schema) that contains 8 boolean fields. Stored directly, this would occupy 8*8 = 64 Bits in our database of which 56 bits would be wasted and unused.

We will use a custom Datatype to compress all 8 booleans into a smallint.

Small int is the smallest numeric datatype in postgres and occupies 2 instead of 4 Bytes. The range of numbers we can express by default is from -32768 to 32767. Other SQL variants might have even something like a tinyint to further reduce the field size but that is of little relevance to us.

We will also use virtual fields to capture the booleans without persisting them directly, using our custom datatype instead.

Objective: Creating Schema with Virtual Fields and Custom Datatype (1 point)

Lets generate a new live resource Item via a phoenix genernator.

mix phx.gen.live Items Item items name:string attributes:integer

Warning! If you have followed along the video you might have already created the Item resources. In this case create a new migration and add the attributes field instead. You don't need to recreate the :categories Enum type field from the video.

mix ecto.gen.migration add_attributes_to_items

Don't forget to change the datatype to smallint before you commit and migrate:

defmodule App.Repo.Migrations.CreateItems do
  use Ecto.Migration

  def change do
    create table(:items) do
      add :name, :string
      add :attributes, :smallint

      timestamps(type: :utc_datetime)
    end
  end
end

Back in our schema we want to create a bunch of virtual fields and change the attributes field to our custom type:

defmodule App.Items.Item do
  use Ecto.Schema
  import Ecto.Changeset

  schema "items" do
    field :name, :string
    field :attributes, App.Items.AttributesType

    Enum.each(1..8, &field(String.to_atom("attr#{&1}"), :boolean, default: false, virtual: true))

    timestamps(type: :utc_datetime)
  end

  @required ~w(name)a
  @optional ~w(attr1 attr2 attr3 attr4 attr5 attr6 attr7 attr8)a
  def changeset(item, attrs) do
    item
    |> cast(attrs, @required ++ @optional)
    |> validate_required(@required)
    |> prepare_changes(fn changeset ->
        # TODO: extract the values from the individual attr fields and merge them into your integer via `put_change`

        # this will likely change/removed once you have implemented TODO
        put_change(changeset, :attributes, 0)
      end)
  end
end

We have used a virtual field before! Checkout the user schema to see how we used it to enter the password, while we didn't persist the password and persisted a password_hash instead. Virtual fields behave just like regular fields except that we don't have a matching entry in the migration and thus database. They are intended for preprocessing input.

prepare_changes/2 is a very convenient function to only do something when we actually insert/update/delete a record as opposed to do it everytime a field is changed. You should definately look it up in the reference and learn more.

put_change/3 allows us to set the value of a field on the backend, bypassing all validations.

Objective: prepare_changes/3 correctly merges the virtual field values (2 points)

To get this point you will have to succefully merge all the booleans and convert it into an integer representing the binary form of the merged booleans. For example:

101 = 2^2 * 1 + 2^1 * 0 + 2^0 * 0 = 5

Use any technique you have learned in CSCI 204 to achieve this objective. You may also explore the elixir documentation for a function that allows you to directly convert a binary string into an integer. Any valid approach will count a point.

Objective: Custom Datatype (2 points)

Now its time to focus our attention back to the CustomDatatype we have yet to implement. Here is a starting template. You will need to think carefully and complete the cast, load, and dump rules. Remember you can use multiple function clauses and consult the Ecto Documentation for an example of custom datatypes.

defmodule App.Items.AttributesType do
  use Ecto.Type

  def type, do: :integer

  # Provide custom casting rules.
  def cast # TODO

  # When loading data from the database we want to return a map with the attributes and values as key value pairs, for example:
  # %{attr1: false, attr2: true, attr3: false, attr4: true, attr5: false, ...}
  # This means you will need to convert the integer back into booleans.
  def load # TODO

  # When dumping data to the database, we expect an integer
  def dump # TODO
end

Objective: Piecing it together (2 points)

The final points can be earned when you can successfully demonstrate persisting the custom datatype via checkbox fields in your form for the virtual fields that get persisted as the integer through the attributes field.

Remember to also set the value of the virtual fields back to what you extracted from the attributes field so they don't reset to false each time we want to edit an Item.

Here is a screenshot of my final items form: