Add in-memory storage option

[matus-tomlein]

• need to implement the EventStore struct with an implementation that stores the events in memory
• need to implement the SessionStore struct with an implementation that stores the session in memory
• need to remove the SQLiteStorage and references to it and instead pass the custom implementations

[BenjaminDavison]

Matus Tomlein (@matus-tomlein) what is actually involved in this? We've set the sqlite location to :memory: which is one way to apparently make it work. But we're seeing downstream issues that we think are related.

  "failure":{
         "timestamp":"2023-02-07T17:15:05.916507Z",
         "messages":[
            {
               "enrichment":null,
               "message":{
                  "field":"eid",
                  "value":"00000000000000000000000000000000",
                  "expectation":"not a valid UUID"
               }
            }
         ]
      },
      ```
      
      There must be something more to it than :memory: if you've created this issue. If you point us in the right direction we might be able to fix it ourselves. 
      
      I should mention we do get the event, it's just eid sets itself to `00000000000000000000000000000000` which fails validation (this is the pubsub enricher btw)

[matus-tomlein]

Hi BenjaminDavison,

The error that you posted above suggests that the event has an emty event ID (eid) property. I don't understand why this would be caused by running the SQLite in memory – the event ID is assigned before the events are added to the event store, it shouldn't matter what is the event store implementation. It might be useful to look at the full payload of the failed event to see what payload is sent to the collector – there might be other problems with it than what we see from the error.

Anyway, in this issue I meant to implement a simple event queue that does not build on SQLite but just appends tracked events to an in-memory structure (e.g., a list or a dictionary) as they are tracked and removes it from there when they are sent to the collector. No need to use SQLite in this case as the logic should be fairly simple. You can see a similar implementation of the idea in our Java tracker.

We'd be happy to receive external contributions for this if it is something that you are willing to work on. Nevertheless, running SQLite in memory is also a valid approach here although it has a bit more overhead than would be necessary.

[BenjaminDavison]

Thanks for the answer Matus Tomlein (@matus-tomlein) .

We're also confused on why it's setting an eid to 000, for context we have a game client and game server both using the same code base. For our game client we use the normal sqlite file method, but our server instance doesn't allow the creation of any files on it's filesystem (annoying) so we use :memory: . Events from the client are valid, but from the server they all have this error with the only difference we can think of being :memory:.

We will investigate a bit further and see what we come up with.

[BenjaminDavison]

Found the issue on our side, we had a bit of custom code which was zeroing on linux :)