Working with INT8 Vector Type in EF Core
In This Topic
EF Core for SQLite provides support for creating and working with the INT8 vector data type (SQLiteInt8Vector).
This article shows how to create tables with INT8 vector columns and how to work with SQLiteInt8Vector for tasks such as distance calculations, vector characteristics, equality checks, parsing, and conversions.
Prerequisites
SQLite vector functionality is available only through virtual tables provided by the sqlite-vec extension. To work with vectors, you must install the sqlite-vec NuGet package (with prerelease enabled) and load the vec0.dll library into your project.
To obtain vec0.dll, navigate to the sqlite-vec releases page on GitHub. In the Assets section, download and extract the archive corresponding to your operating system.
For EF Core, you can load the obtained vec0.dll into your project in two ways: using the LoadExtension property or the LoadExtension method.
- Load the extension by using the LoadExtension property
string dbFile = "SqlVector.db";
if (!File.Exists(dbFile))
SQLiteConnection.CreateFile("SqlVector.db");
var csb = new SQLiteConnectionStringBuilder();
csb.DataSource = "SqlVector.db";
csb.Version = 3;
csb.EnableLoadExtension = true;
csb.LoadExtension = "vec0.dll";
var conn = new SQLiteConnection(csb.ConnectionString);
conn.Open();
var optionsBuilder = new DbContextOptionsBuilder<SQLiteDbContext>();
optionsBuilder.UseSQLite(conn);
var context = new SQLiteDbContext(optionsBuilder.Options);
- Load the extension by using the LoadExtension method
string dbFile = "SqlVector.db";
if (!File.Exists(dbFile))
SQLiteConnection.CreateFile(dbFile);
var conn = new SQLiteConnection(connectionString);
conn.Open();
conn.LoadExtension("vec0.dll");
var optionsBuilder = new DbContextOptionsBuilder<SQLiteDbContext>();
optionsBuilder.UseSQLite(conn);
var context = new SQLiteDbContext(optionsBuilder.Options);
Creating a Virtual INT8 Vector Table
To create a virtual table with an INT8 vector column, use the following SQL statement:
CREATE VIRTUAL TABLE DOCUMENTS_BINARY USING vec0(EMBEDDING int8[3])
Working with SQLiteInt8Vector
This section describes common operations supported by the SQLiteVectorFunctions helpers and the SQLiteInt8Vector type.
Calculating Vector Distance
To calculate distance between INT8 vectors, use Manhattan, Euclidean, and cosine metrics and the Match operator.
// Manhattan distance
List<SQLiteInt8Vector> result = context.BinaryVectorDocuments.OrderBy(e => SQLiteVectorFunctions.ManhattanDistance(e.Embedding, vector)).Select(d => d.Embedding).Take(1).ToList();
// Euclidean distance
List<SQLiteInt8Vector> result = dbContext.BinaryDocuments.Where(e => SQLiteVectorFunctions.EuclideanDistance(e.Embedding, vector)).Select(d => d.Embedding).Take(1).ToList();
// Cosine distance
List<SQLiteInt8Vector> result = dbContext.BinaryDocuments.Where(e => SQLiteVectorFunctions.CosineDistance(e.Embedding, vector)).Select(d => d.Embedding).Take(1).ToList();
// Match
List<SQLiteInt8Vector> result = dbContext.BinaryDocuments.Where(e => SQLiteVectorFunctions.Match(e.Embedding, vector)).Select(d => d.Embedding).Take(1).ToList();
Getting Vector Length
To retrieve the number of elements in an INT8 vector, use the SQLiteVectorFunctions.Length method.
int result = dbContext.BinaryDocuments.Select(d => SQLiteVectorFunctions.Length(d.Embedding)).FirstOrDefault();
Getting Vector Type
To retrieve the type of a vector, use the SQLiteVectorFunctions.Type method.
string result = dbContext.BinaryDocuments.Select(d => SQLiteVectorFunctions.Type(d.Embedding)).FirstOrDefault();
Checking Equality
To check if two INT8 vectors are equal, use the == operator.
SQLiteInt8Vector vector1 = new SQLiteInt8Vector(new List<SByte> { 10, 20, 30 });
SQLiteInt8Vector vector2 = new SQLiteInt8Vector(new List<SByte> { 10, 20, 30 });
if (vector1 == vector2)
Console.WriteLine("vector1 and vector2 are the same");
Checking Inequality
To check if two INT8 vectors are not equal, use the != operator.
SQLiteInt8Vector vector1 = new SQLiteInt8Vector(new List<SByte> { 10, 20, 30 });
SQLiteInt8Vector vector2 = new SQLiteInt8Vector(new List<SByte> { 10, 20, 30 });
if (vector1 != vector2)
Console.WriteLine("vector1 and vector2 are not the same");
Parsing from String
Using TryParse
To safely convert a string to an INT8 vector, use the TryParse method.
SQLiteInt8Vector vector = new SQLiteInt8Vector(new List<SByte> { 10, 20, 30 });
string stringVector = "[10,20,30]";
if(SQLiteInt8Vector.TryParse(stringVector, out vector)) {
Console.WriteLine("You can parse stringVector");
}
Using Parse
To directly convert a string to an INT8 vector, use the Parse method.
string stringVector = "[10,20,30]";
SQLiteInt8Vector newVector = SQLiteInt8Vector.Parse(stringVector);
Converting Vector to String
To convert an INT8 vector to its string representation, use the ToString method.
SQLiteInt8Vector vector = new SQLiteInt8Vector(new List<SByte> { 10, 20, 30 });
string stringVector = vector.ToString();
Getting Hash Code
To retrieve the hash code of an INT8 vector, use the GetHashCode method.
SQLiteInt8Vector vector = new SQLiteInt8Vector(new List<SByte> { 10, 20, 30 });
int hashCode = vector.GetHashCode();