c# – Asynchronize (part of) a synchrone library

by

To contextualize my issue, here his what I’m working on: In order to explore new .Net MAUI, I create a “simple” project to draw lotteries numbers based on past statistics. I chose to develop a library for my business logic, and then use it in another .Net MAUI solution. So for now forget about .Net MAUI (expect that’s the long term goal).

I create a VS solution of 2 projects:

  • first one a simple console app (that will use the second one)
  • second one the library project

All my code is in the library project, the console one only references it and calls it.

In my library project, I want to hide everything, so I made it internalexcepted for my interface.

To allow using the library from external, I code an API with lot of method to manipulate my library’s objects.

using Loto.Converters;
using Microsoft.Extensions.DependencyInjection;

namespace Loto
{
public interface ILotteryAPI
{
    public ILotteryManager Manager => HostUtils.MyHost.Services.GetRequiredService<ILotteryManager>();        

        #region Converter
        IToStatsConverter? CreateDummyConverter(Type converterType)
            => CreateConverter<object>(converterType, null, false);
        IToStatsConverter? CreateConverter<TConv, T>(T data, bool toRegister = true) where TConv : IToStatsConverter where T : class
            => CreateConverter(typeof(TConv), Guid.Empty, data, toRegister);
        IToStatsConverter? CreateConverter<TConv, T>(Guid lotteryId, T data, bool toRegister = true) where TConv : IToStatsConverter where T : class
            => CreateConverter(typeof(TConv), lotteryId, data, toRegister);

        IToStatsConverter? CreateConverter<T>(Type converterType, T? data, bool toRegister = true) where T : class
            => CreateConverter(converterType, Guid.Empty, data, toRegister);
        IToStatsConverter? CreateConverter<T>(Type converterType, Guid lotteryId, T? data, bool toRegister = true) where T : class
            => CreateConverter(converterType, lotteryId, (data == null ? new T[] { } : new T[] { data }) as IEnumerable<T>, toRegister);

        IToStatsConverter? CreateConverter<TConv, T>(IEnumerable<T>? datas, bool toRegister = true) where TConv : IToStatsConverter where T : class
            => CreateConverter(typeof(TConv), Guid.Empty, datas, toRegister);
        IToStatsConverter? CreateConverter<TConv, T>(Guid lotteryId, IEnumerable<T>? datas, bool toRegister = true) where TConv : IToStatsConverter where T : class
            => CreateConverter(typeof(TConv), lotteryId, datas, toRegister);

        IToStatsConverter? CreateConverter<T>(Type converterType, IEnumerable<T>? datas, bool toRegister = true) where T : class
            => CreateConverter(converterType, Guid.Empty, datas, toRegister);

        IToStatsConverter? CreateConverter<T>(Type converterType, Guid lotteryId, IEnumerable<T>? datas, bool toRegister = true) where T : class
            => Manager.CreateConverter(converterType, lotteryId, datas, toRegister);

        IToStatsConverter? GetConverter(ILottery lottery) => Manager.FindFirstConverter(lottery);
        IToStatsConverter? GetConverter<T>(ILottery lottery) => Manager.FindFirstConverter<T>(lottery);
        IToStatsConverter? GetConverter<T>(ILottery lottery, IEnumerable<T> datas) where T : class => Manager.GetConverter(lottery, datas?.FirstOrDefault() ?? null);
        IToStatsConverter? GetConverter<T>(ILottery lottery, T data) where T : class => Manager.GetConverter(lottery, data);
        IToStatsConverter? GetConverter<T>(Type converterType, IEnumerable<T> sources, bool canCreate = false) where T : class;

        IToStatsConverter? GetConverter<T>(Type converterType, T source, bool canCreate = false) where T : class
            => GetConverter(converterType, new T[] { source }, canCreate);
        IToStatsConverter? GetConverter<T>(IEnumerable<T> sources, bool canCreate = false) where T : class;

        IToStatsConverter? GetConverter<T>(T source, bool canCreate = false) where T : class
            => GetConverter(new T[] { source }, canCreate);
        IToStatsConverter? GetConverter<TConv, T>(IEnumerable<T> sources, bool canCreate = false) where TConv : IToStatsConverter where T : class
            => GetConverter(typeof(TConv), sources, canCreate);
        IToStatsConverter? GetConverter<TConv, T>(T source, bool canCreate = false) where TConv : IToStatsConverter where T : class
            => GetConverter<TConv, T>(new T[] { source }, canCreate);

        List<IToStatsConverter> GetConverters();
        List<IToStatsConverter>? GetConverters(ILottery lottery) => Manager.GetConverters(lottery);

        bool AddConverter(IToStatsConverter converter) => Manager.RegisterConverter(converter);

        bool AddConverterTo(IToStatsConverter converter, ILottery lottery);
        #endregion

        #region Lottery
        ILottery CreateLottery(Guid id, List<IGrid> grids, string? name = default, bool registering = true, bool checkNoEmptyGrids = true);
        ILottery CreateLottery(List<IGrid> grids, string? name = default, bool registering = true, bool checkNoEmptyGrids = true)
            => CreateLottery(Guid.NewGuid(), grids, name, registering, checkNoEmptyGrids);
        ILottery CreateLottery(bool registering = true, bool checkNoEmptyGrids = true)
            => CreateLottery(Guid.NewGuid(), new List<IGrid>(), null, registering, checkNoEmptyGrids);

        ILottery? GetLottery() => Manager.GetLottery();
        ILottery? GetLottery(Guid guid) => Manager.GetLottery(guid);
        ILottery? GetLottery(ILottery other) => Manager.GetLottery(other);
        ILottery? GetLottery(IToStatsConverter converter) => GetLottery(converter.Lottery);
        ILottery? GetLottery<T>(T? source = null) where T : class;
        ILottery? GetLottery<T>(IEnumerable<T> sources) where T : class 
            => GetLottery(sources?.FirstOrDefault() ?? null);
        ILottery? GetLottery(IGrid grid);

        List<ILottery> GetLotteries() => Manager.LotteryCollection.Keys.ToList();

        bool HaveGrid(ILottery lottery, IGrid grid);
        #endregion

        #region Grid
        IGrid CreateGrid(int start, int end, int numberOfDraws, Guid id, Guid lotteryId, string name = null)
            => Manager.CreateGrid(start, end, numberOfDraws, id, lotteryId, name);
        IGrid CreateGrid(int start, int end, int numberOfDraws, Guid id, ILottery? lottery = null, string name = null)
           => CreateGrid(start, end, numberOfDraws, id, lottery?.Id ?? Guid.Empty, name);
        IGrid CreateGrid(int start, int end, int numberOfDraws, Guid lotteryId, string name = null)
            => CreateGrid(start, end, numberOfDraws, Guid.NewGuid(), lotteryId, name);
        IGrid CreateGrid(int start, int end, int numberOfDraws, ILottery? lottery = null, string name = null)
            => CreateGrid(start, end, numberOfDraws, Guid.NewGuid(), lottery?.Id ?? Guid.Empty, name);

        IGrid? GetGrid(Guid gridGuid) => Manager.GetGrid(gridGuid);
        IGrid? GetGrid(IGridStats gridStats) => Manager.GetGrid(gridStats);

        bool AddGrid(ILottery lottery, IGrid grid) => Manager.AddGrid(lottery, grid);

        bool AddGrids(ILottery lottery, List<IGrid> grids);
        #endregion

        #region Stats
        ILotteryStats? GetStats(ILottery lottery) => Manager.GetStats(lottery);
        ILotteryStats? GetStats(string filename) => Manager.GetStats(filename);

        bool AddStats(ILottery lottery, IEnumerable<ILotteryStats> lotteriesStats)
            => Manager.AddStats(lottery, lotteriesStats);
        bool AddStats(ILottery lottery, ILotteryStats lotteryStats)
            => Manager.AddStats(lottery, lotteryStats);
        bool AddStats(IToStatsConverter converter)    
            => AddStats(converter.Lottery, converter.ConvertAll() as IEnumerable<ILotteryStats>);
        bool AddStats<T>(ILottery lottery, T data) where T : class;

        int[] GetRawStats(IGridStats gridStats);
    
        IGridStats? GetConcatStats(IGrid grid, string? name = null);

        List<int[]> GetRawStats(ILotteryStats lotteryStats);
        List<int[]> GetRawStats(ILottery lottery);
        List<int[]> GetRawStats<T>(ILottery lottery, T data, bool dataRegisteringNeeded = true) where T : class;

        ILotteryStats Concat( ILotteryStats left, ILotteryStats? right);

        FileInfo[]? GetAllStatsFileFrom(string directoryPath, string? ext = null);
    
        bool SerializeStats(ILotteryStats lotteryStats, string? filename = null) => SerializationUtils.Serialize(lotteryStats, filename);
        bool SerializeStats(ILottery lottery, string? filename = null) => SerializationUtils.Serialize(GetStats(lottery), filename == null ? lottery.Name : filename);

        async Task<bool> SerializeStatsAsync(ILotteryStats lotteryStats, string? filename = null) => await SerializationUtils.SerializeAsync(lotteryStats, filename);
        async Task<bool> SerializeStatsAsync(ILottery lottery, string? filename = null) => await SerializationUtils.SerializeAsync(GetStats(lottery), filename == null ? lottery.Name : filename);
        #endregion

        #region Sources
        int AddSource<T>(IToStatsConverter converter, T data) where T : class;
        int AddSource<T>(ILottery lottery, T data) where T : class;
        int AddSource<T>(T data, bool canCreate = false) where T : class 
            => AddSource(GetConverter(data, canCreate), data);

        bool AddSources<T>(IToStatsConverter converter, IEnumerable<T> datas) where T : class;
        bool AddSources<T>(ILottery lottery, IEnumerable<T> datas) where T : class;
        bool AddSources<T>(IEnumerable<T> datas, bool canCreate = false) where T : class 
            => AddSources(GetConverter(datas, canCreate), datas);
        #endregion

        #region Saving
        bool Load(bool forceLoading = false) => SerializationUtils.Load(forceLoading);
        bool Load(string filename, bool forceLoading = false) 
            => SerializationUtils.Load(filename, forceLoading);
        bool Save() => SerializationUtils.Save();
        bool Save(string filename) => SerializationUtils.Save(filename);

        async Task<bool> LoadAsync(bool forceLoading = false) 
            => await SerializationUtils.LoadAsync(forceLoading);
        async Task<bool> LoadAsync(string filename, bool forceLoading = false) 
            => await SerializationUtils.LoadAsync(filename, forceLoading);
        async Task<bool> SaveAsync() => await SerializationUtils.SaveAsync();
        async Task<bool> SaveAsync(string filename) => await SerializationUtils.SaveAsync(filename);
        #endregion

        bool InitFolders() 
            => SerializationUtils.InitSavesFolder() && SerializationUtils.InitLotoStatsFolder();

        List<int[]> Draw(ILottery lottery);        

        string SelectionToString(ILottery lottery);        
    }
}

For my data structure, I use 2 Dictionary

Dictionary<ILottery, ICachedLotteryStats>? _lotteryCollection;
Dictionary<Type, List<IToStatsConverter>>? _converters;

I persist that data via Json (de)serialization of .Net Text.Json (Save/Load).

Before going to .Net MAUI side, I wanted to make my Save/Load asynchronous as it may take time and don’t want to freeze my next to be UI.

I created 2 extra method LoadAsync and SaveAsyncand in order to lock them up (to prevent corrupted save if ever LoadAsync/SaveAsync thread intertwined) I guard them under a SemaphoreSlim:

private static readonly SemaphoreSlim _lotoSemaphore = new SemaphoreSlim(1, 1);
public static SemaphoreSlim LotoSemaphore => _lotoSemaphore;

EDIT : example of Load() vs LoadAsync()

public static bool Load(string saveFile, bool forceLoading = false)
{
    _onSaveLoad = true;
    try
    {
        if (_hasLoaded && !forceLoading)
        {
            Log.Debug("Already loaded before, skip another useless loading then");
            return true;
        }

        if (!IsValidPath(saveFile))
        {
            Log.Error("Load failed! {path} is not a valid path", saveFile);
            return false;
        }

        if (!File.Exists(saveFile))
        {
            Log.Warning("Cannot Load, no save file found at {path}", saveFile);
            return false;
        }

        var jsonLines = File.ReadLines(saveFile);
        if (jsonLines.Count() != 2)
        {
            Log.Error("Save file should have only 2 lines, one for each dictionnary object to deserialize!");
            return false;
        }
           
        var lotoCol = JsonSerializer.Deserialize<Dictionary<ILottery, ICachedLotteryStats>>(jsonLines.First(), Options);
        if (lotoCol != null) Loto_API.API.Manager.LotteryCollection = lotoCol; else { _hasLoaded = true; return false; }

        var convs = JsonSerializer.Deserialize<Dictionary<Type, List<IToStatsConverter>>>(jsonLines.Last(), Options);
        if (convs != null) { Loto_API.API.Manager.Converters = convs; _hasLoaded = true; return true; }
      
        Log.Error("Deserialization returned null dictionary(ies)! {path} Save file may be corrupted.", saveFile);
        return false;
    }
    finally { _onSaveLoad = false; }
}

Async version :

public static async Task<bool> LoadAsync(string saveFile, bool forceLoading = false)
{
    await Loto_API.ConstUtils.LotoSemaphore.WaitAsync();
    try
    {
        _onSaveLoad = true;
         if (_hasLoaded && !forceLoading)
        {
            Log.Debug("Already loaded before, skip another useless loading then");
            return true;
        }
            
        if (!IsValidPath(saveFile))
        {
            Log.Error("Load failed! {path} is not a valid path", saveFile);
            return false;
        }

        if (!File.Exists(saveFile))
        {
            Log.Warning("Cannot Load, no save file found at {path}", saveFile);
            return false;
        }
            
        var jsonLines = await File.ReadAllLinesAsync(saveFile);//ASYNC

        if (jsonLines.Count() != 2)
        {
            Log.Error("Save file should have only 2 lines, one for each dictionnary object to deserialize!");
            return false;
        }

        var lotoCol = await GetValueAsync<Dictionary<ILottery, ICachedLotteryStats>>(jsonLines.First());//ASYNC
        if (lotoCol != null) await Loto_API.API.Manager.LotteryCollection = lotoCol; else { _hasLoaded = true; return false; }
           
        var convs = await GetValueAsync<Dictionary<Type, List<IToStatsConverter>>>(jsonLines.Last());//ASYNC
        if (convs != null) { await Loto_API.API.Manager.Converters = convs; _hasLoaded = true; return true; }
    
        Log.Error("Deserialization returned null dictionary(ies)! {path} Save file may be corrupted.", saveFile);
        return false;
    }
    finally
    {
        _onSaveLoad = false;
        Loto_API.ConstUtils.LotoSemaphore.Release();
    }
}

with this helper function :

public static async Task<TValue> GetValueAsync<TValue>(string json)
{
    try
    {
        using (var stream = new MemoryStream(Encoding.UTF8.GetBytes(json)))
        {
            return await JsonSerializer.DeserializeAsync<TValue>(stream, Options);
        }
    }
    catch (Exception e)
    {
        Log.Error(e, "Error while trying to deserialize asynchronously a json string to a {valueType} object", nameof(TValue));
        return default;
    }
}

The issue, from my Console project using my library:

string statPath = Path.Combine(Loto_API.ConstUtils.STATS_FOLDER, "EuroMillion");

var statsFiles = Loto_API.API.GetAllStatsFileFrom(statPath, "*.csv");
Loto_API.API.LoadAsync();
Loto_API.API.LoadAsync();

Loto_API.API.AddSources(statsFiles, true);/// TO TEST

Log.Information(Loto_API.API.SelectionToString(Loto_API.API.GetLottery()));

Loto_API.API.SaveAsync();
Loto_API.API.LoadAsync();

is that when I call some API method, they may use one or both of my 2 Dictonary objects; and I don’t know if load/save is done or not.

So I tried to guard my two Dictonary under the same SemaphoreSlimbut my LoadAsync/SaveAsync methods, of course, try to access them too.

I starting to create a new API class decorator implementing my ILotteryAPIusing the standard synchrone API as the decorated one, and surrounding all method with the semaphore guard.

example:

private async Task<FileInfo[]?> _GetAllStatsFileFromAsync(string directoryPath, string? ext = null)
{
    await _before.Invoke();
    try
    {
        Console.WriteLine(" /!\ Inside Decorated API!");
        return _api.GetAllStatsFileFrom(directoryPath, ext);
    }
    finally
    {
        await _after.Invoke();
    }
}
public override FileInfo[]? GetAllStatsFileFrom(string directoryPath, string? ext = null)
    => _GetAllStatsFileFromAsync(directoryPath, ext).Result;

But I think it goes the wrong / too complex way now.

So to sum up my question:

  • Given a library which exposes its methods via an API (synchrone)
  • The library handle persisting data via Json Serialization, which may took time
  • The library expose asynchronous version of Saving / Loading data under a semaphore lock

How to prevent manipulation/access of the library’s datas (hided behind it’s API methods) while asynchrone Saving/Loading is on the way on them (and so keeping Saving/Loading to modify/access them)?

Thank you.

Leave a Comment