To contextualize my issue, here his what I’m working on: In order to explore new .Net MAUI, I create a “simple” project to draw lotteries numbers based on past statistics. I chose to develop a library for my business logic, and then use it in another .Net MAUI solution. So for now forget about .Net MAUI (expect that’s the long term goal).
I create a VS solution of 2 projects:
- first one a simple console app (that will use the second one)
- second one the library project
All my code is in the library project, the console one only references it and calls it.
In my library project, I want to hide everything, so I made it internal
excepted for my interface
.
To allow using the library from external, I code an API with lot of method to manipulate my library’s objects.
using Loto.Converters;
using Microsoft.Extensions.DependencyInjection;
namespace Loto
{
public interface ILotteryAPI
{
public ILotteryManager Manager => HostUtils.MyHost.Services.GetRequiredService<ILotteryManager>();
#region Converter
IToStatsConverter? CreateDummyConverter(Type converterType)
=> CreateConverter<object>(converterType, null, false);
IToStatsConverter? CreateConverter<TConv, T>(T data, bool toRegister = true) where TConv : IToStatsConverter where T : class
=> CreateConverter(typeof(TConv), Guid.Empty, data, toRegister);
IToStatsConverter? CreateConverter<TConv, T>(Guid lotteryId, T data, bool toRegister = true) where TConv : IToStatsConverter where T : class
=> CreateConverter(typeof(TConv), lotteryId, data, toRegister);
IToStatsConverter? CreateConverter<T>(Type converterType, T? data, bool toRegister = true) where T : class
=> CreateConverter(converterType, Guid.Empty, data, toRegister);
IToStatsConverter? CreateConverter<T>(Type converterType, Guid lotteryId, T? data, bool toRegister = true) where T : class
=> CreateConverter(converterType, lotteryId, (data == null ? new T[] { } : new T[] { data }) as IEnumerable<T>, toRegister);
IToStatsConverter? CreateConverter<TConv, T>(IEnumerable<T>? datas, bool toRegister = true) where TConv : IToStatsConverter where T : class
=> CreateConverter(typeof(TConv), Guid.Empty, datas, toRegister);
IToStatsConverter? CreateConverter<TConv, T>(Guid lotteryId, IEnumerable<T>? datas, bool toRegister = true) where TConv : IToStatsConverter where T : class
=> CreateConverter(typeof(TConv), lotteryId, datas, toRegister);
IToStatsConverter? CreateConverter<T>(Type converterType, IEnumerable<T>? datas, bool toRegister = true) where T : class
=> CreateConverter(converterType, Guid.Empty, datas, toRegister);
IToStatsConverter? CreateConverter<T>(Type converterType, Guid lotteryId, IEnumerable<T>? datas, bool toRegister = true) where T : class
=> Manager.CreateConverter(converterType, lotteryId, datas, toRegister);
IToStatsConverter? GetConverter(ILottery lottery) => Manager.FindFirstConverter(lottery);
IToStatsConverter? GetConverter<T>(ILottery lottery) => Manager.FindFirstConverter<T>(lottery);
IToStatsConverter? GetConverter<T>(ILottery lottery, IEnumerable<T> datas) where T : class => Manager.GetConverter(lottery, datas?.FirstOrDefault() ?? null);
IToStatsConverter? GetConverter<T>(ILottery lottery, T data) where T : class => Manager.GetConverter(lottery, data);
IToStatsConverter? GetConverter<T>(Type converterType, IEnumerable<T> sources, bool canCreate = false) where T : class;
IToStatsConverter? GetConverter<T>(Type converterType, T source, bool canCreate = false) where T : class
=> GetConverter(converterType, new T[] { source }, canCreate);
IToStatsConverter? GetConverter<T>(IEnumerable<T> sources, bool canCreate = false) where T : class;
IToStatsConverter? GetConverter<T>(T source, bool canCreate = false) where T : class
=> GetConverter(new T[] { source }, canCreate);
IToStatsConverter? GetConverter<TConv, T>(IEnumerable<T> sources, bool canCreate = false) where TConv : IToStatsConverter where T : class
=> GetConverter(typeof(TConv), sources, canCreate);
IToStatsConverter? GetConverter<TConv, T>(T source, bool canCreate = false) where TConv : IToStatsConverter where T : class
=> GetConverter<TConv, T>(new T[] { source }, canCreate);
List<IToStatsConverter> GetConverters();
List<IToStatsConverter>? GetConverters(ILottery lottery) => Manager.GetConverters(lottery);
bool AddConverter(IToStatsConverter converter) => Manager.RegisterConverter(converter);
bool AddConverterTo(IToStatsConverter converter, ILottery lottery);
#endregion
#region Lottery
ILottery CreateLottery(Guid id, List<IGrid> grids, string? name = default, bool registering = true, bool checkNoEmptyGrids = true);
ILottery CreateLottery(List<IGrid> grids, string? name = default, bool registering = true, bool checkNoEmptyGrids = true)
=> CreateLottery(Guid.NewGuid(), grids, name, registering, checkNoEmptyGrids);
ILottery CreateLottery(bool registering = true, bool checkNoEmptyGrids = true)
=> CreateLottery(Guid.NewGuid(), new List<IGrid>(), null, registering, checkNoEmptyGrids);
ILottery? GetLottery() => Manager.GetLottery();
ILottery? GetLottery(Guid guid) => Manager.GetLottery(guid);
ILottery? GetLottery(ILottery other) => Manager.GetLottery(other);
ILottery? GetLottery(IToStatsConverter converter) => GetLottery(converter.Lottery);
ILottery? GetLottery<T>(T? source = null) where T : class;
ILottery? GetLottery<T>(IEnumerable<T> sources) where T : class
=> GetLottery(sources?.FirstOrDefault() ?? null);
ILottery? GetLottery(IGrid grid);
List<ILottery> GetLotteries() => Manager.LotteryCollection.Keys.ToList();
bool HaveGrid(ILottery lottery, IGrid grid);
#endregion
#region Grid
IGrid CreateGrid(int start, int end, int numberOfDraws, Guid id, Guid lotteryId, string name = null)
=> Manager.CreateGrid(start, end, numberOfDraws, id, lotteryId, name);
IGrid CreateGrid(int start, int end, int numberOfDraws, Guid id, ILottery? lottery = null, string name = null)
=> CreateGrid(start, end, numberOfDraws, id, lottery?.Id ?? Guid.Empty, name);
IGrid CreateGrid(int start, int end, int numberOfDraws, Guid lotteryId, string name = null)
=> CreateGrid(start, end, numberOfDraws, Guid.NewGuid(), lotteryId, name);
IGrid CreateGrid(int start, int end, int numberOfDraws, ILottery? lottery = null, string name = null)
=> CreateGrid(start, end, numberOfDraws, Guid.NewGuid(), lottery?.Id ?? Guid.Empty, name);
IGrid? GetGrid(Guid gridGuid) => Manager.GetGrid(gridGuid);
IGrid? GetGrid(IGridStats gridStats) => Manager.GetGrid(gridStats);
bool AddGrid(ILottery lottery, IGrid grid) => Manager.AddGrid(lottery, grid);
bool AddGrids(ILottery lottery, List<IGrid> grids);
#endregion
#region Stats
ILotteryStats? GetStats(ILottery lottery) => Manager.GetStats(lottery);
ILotteryStats? GetStats(string filename) => Manager.GetStats(filename);
bool AddStats(ILottery lottery, IEnumerable<ILotteryStats> lotteriesStats)
=> Manager.AddStats(lottery, lotteriesStats);
bool AddStats(ILottery lottery, ILotteryStats lotteryStats)
=> Manager.AddStats(lottery, lotteryStats);
bool AddStats(IToStatsConverter converter)
=> AddStats(converter.Lottery, converter.ConvertAll() as IEnumerable<ILotteryStats>);
bool AddStats<T>(ILottery lottery, T data) where T : class;
int[] GetRawStats(IGridStats gridStats);
IGridStats? GetConcatStats(IGrid grid, string? name = null);
List<int[]> GetRawStats(ILotteryStats lotteryStats);
List<int[]> GetRawStats(ILottery lottery);
List<int[]> GetRawStats<T>(ILottery lottery, T data, bool dataRegisteringNeeded = true) where T : class;
ILotteryStats Concat( ILotteryStats left, ILotteryStats? right);
FileInfo[]? GetAllStatsFileFrom(string directoryPath, string? ext = null);
bool SerializeStats(ILotteryStats lotteryStats, string? filename = null) => SerializationUtils.Serialize(lotteryStats, filename);
bool SerializeStats(ILottery lottery, string? filename = null) => SerializationUtils.Serialize(GetStats(lottery), filename == null ? lottery.Name : filename);
async Task<bool> SerializeStatsAsync(ILotteryStats lotteryStats, string? filename = null) => await SerializationUtils.SerializeAsync(lotteryStats, filename);
async Task<bool> SerializeStatsAsync(ILottery lottery, string? filename = null) => await SerializationUtils.SerializeAsync(GetStats(lottery), filename == null ? lottery.Name : filename);
#endregion
#region Sources
int AddSource<T>(IToStatsConverter converter, T data) where T : class;
int AddSource<T>(ILottery lottery, T data) where T : class;
int AddSource<T>(T data, bool canCreate = false) where T : class
=> AddSource(GetConverter(data, canCreate), data);
bool AddSources<T>(IToStatsConverter converter, IEnumerable<T> datas) where T : class;
bool AddSources<T>(ILottery lottery, IEnumerable<T> datas) where T : class;
bool AddSources<T>(IEnumerable<T> datas, bool canCreate = false) where T : class
=> AddSources(GetConverter(datas, canCreate), datas);
#endregion
#region Saving
bool Load(bool forceLoading = false) => SerializationUtils.Load(forceLoading);
bool Load(string filename, bool forceLoading = false)
=> SerializationUtils.Load(filename, forceLoading);
bool Save() => SerializationUtils.Save();
bool Save(string filename) => SerializationUtils.Save(filename);
async Task<bool> LoadAsync(bool forceLoading = false)
=> await SerializationUtils.LoadAsync(forceLoading);
async Task<bool> LoadAsync(string filename, bool forceLoading = false)
=> await SerializationUtils.LoadAsync(filename, forceLoading);
async Task<bool> SaveAsync() => await SerializationUtils.SaveAsync();
async Task<bool> SaveAsync(string filename) => await SerializationUtils.SaveAsync(filename);
#endregion
bool InitFolders()
=> SerializationUtils.InitSavesFolder() && SerializationUtils.InitLotoStatsFolder();
List<int[]> Draw(ILottery lottery);
string SelectionToString(ILottery lottery);
}
}
For my data structure, I use 2 Dictionary
Dictionary<ILottery, ICachedLotteryStats>? _lotteryCollection;
Dictionary<Type, List<IToStatsConverter>>? _converters;
I persist that data via Json (de)serialization of .Net Text.Json (Save/Load).
Before going to .Net MAUI side, I wanted to make my Save/Load asynchronous as it may take time and don’t want to freeze my next to be UI.
I created 2 extra method LoadAsync
and SaveAsync
and in order to lock them up (to prevent corrupted save if ever LoadAsync
/SaveAsync
thread intertwined) I guard them under a SemaphoreSlim
:
private static readonly SemaphoreSlim _lotoSemaphore = new SemaphoreSlim(1, 1);
public static SemaphoreSlim LotoSemaphore => _lotoSemaphore;
EDIT : example of Load()
vs LoadAsync()
public static bool Load(string saveFile, bool forceLoading = false)
{
_onSaveLoad = true;
try
{
if (_hasLoaded && !forceLoading)
{
Log.Debug("Already loaded before, skip another useless loading then");
return true;
}
if (!IsValidPath(saveFile))
{
Log.Error("Load failed! {path} is not a valid path", saveFile);
return false;
}
if (!File.Exists(saveFile))
{
Log.Warning("Cannot Load, no save file found at {path}", saveFile);
return false;
}
var jsonLines = File.ReadLines(saveFile);
if (jsonLines.Count() != 2)
{
Log.Error("Save file should have only 2 lines, one for each dictionnary object to deserialize!");
return false;
}
var lotoCol = JsonSerializer.Deserialize<Dictionary<ILottery, ICachedLotteryStats>>(jsonLines.First(), Options);
if (lotoCol != null) Loto_API.API.Manager.LotteryCollection = lotoCol; else { _hasLoaded = true; return false; }
var convs = JsonSerializer.Deserialize<Dictionary<Type, List<IToStatsConverter>>>(jsonLines.Last(), Options);
if (convs != null) { Loto_API.API.Manager.Converters = convs; _hasLoaded = true; return true; }
Log.Error("Deserialization returned null dictionary(ies)! {path} Save file may be corrupted.", saveFile);
return false;
}
finally { _onSaveLoad = false; }
}
Async version :
public static async Task<bool> LoadAsync(string saveFile, bool forceLoading = false)
{
await Loto_API.ConstUtils.LotoSemaphore.WaitAsync();
try
{
_onSaveLoad = true;
if (_hasLoaded && !forceLoading)
{
Log.Debug("Already loaded before, skip another useless loading then");
return true;
}
if (!IsValidPath(saveFile))
{
Log.Error("Load failed! {path} is not a valid path", saveFile);
return false;
}
if (!File.Exists(saveFile))
{
Log.Warning("Cannot Load, no save file found at {path}", saveFile);
return false;
}
var jsonLines = await File.ReadAllLinesAsync(saveFile);//ASYNC
if (jsonLines.Count() != 2)
{
Log.Error("Save file should have only 2 lines, one for each dictionnary object to deserialize!");
return false;
}
var lotoCol = await GetValueAsync<Dictionary<ILottery, ICachedLotteryStats>>(jsonLines.First());//ASYNC
if (lotoCol != null) await Loto_API.API.Manager.LotteryCollection = lotoCol; else { _hasLoaded = true; return false; }
var convs = await GetValueAsync<Dictionary<Type, List<IToStatsConverter>>>(jsonLines.Last());//ASYNC
if (convs != null) { await Loto_API.API.Manager.Converters = convs; _hasLoaded = true; return true; }
Log.Error("Deserialization returned null dictionary(ies)! {path} Save file may be corrupted.", saveFile);
return false;
}
finally
{
_onSaveLoad = false;
Loto_API.ConstUtils.LotoSemaphore.Release();
}
}
with this helper function :
public static async Task<TValue> GetValueAsync<TValue>(string json)
{
try
{
using (var stream = new MemoryStream(Encoding.UTF8.GetBytes(json)))
{
return await JsonSerializer.DeserializeAsync<TValue>(stream, Options);
}
}
catch (Exception e)
{
Log.Error(e, "Error while trying to deserialize asynchronously a json string to a {valueType} object", nameof(TValue));
return default;
}
}
The issue, from my Console project using my library:
string statPath = Path.Combine(Loto_API.ConstUtils.STATS_FOLDER, "EuroMillion");
var statsFiles = Loto_API.API.GetAllStatsFileFrom(statPath, "*.csv");
Loto_API.API.LoadAsync();
Loto_API.API.LoadAsync();
Loto_API.API.AddSources(statsFiles, true);/// TO TEST
Log.Information(Loto_API.API.SelectionToString(Loto_API.API.GetLottery()));
Loto_API.API.SaveAsync();
Loto_API.API.LoadAsync();
is that when I call some API method, they may use one or both of my 2 Dictonary
objects; and I don’t know if load/save is done or not.
So I tried to guard my two Dictonary
under the same SemaphoreSlim
but my LoadAsync
/SaveAsync
methods, of course, try to access them too.
I starting to create a new API class decorator implementing my ILotteryAPI
using the standard synchrone API as the decorated one, and surrounding all method with the semaphore guard.
example:
private async Task<FileInfo[]?> _GetAllStatsFileFromAsync(string directoryPath, string? ext = null)
{
await _before.Invoke();
try
{
Console.WriteLine(" /!\ Inside Decorated API!");
return _api.GetAllStatsFileFrom(directoryPath, ext);
}
finally
{
await _after.Invoke();
}
}
public override FileInfo[]? GetAllStatsFileFrom(string directoryPath, string? ext = null)
=> _GetAllStatsFileFromAsync(directoryPath, ext).Result;
But I think it goes the wrong / too complex way now.
So to sum up my question:
- Given a library which exposes its methods via an API (synchrone)
- The library handle persisting data via Json Serialization, which may took time
- The library expose asynchronous version of Saving / Loading data under a semaphore lock
How to prevent manipulation/access of the library’s datas (hided behind it’s API methods) while asynchrone Saving/Loading is on the way on them (and so keeping Saving/Loading to modify/access them)?
Thank you.