Data Engine Data Engine protocol schemas
Data Engine Protocol
Defines the standard interface for data persistence engines in ObjectStack.
This protocol abstracts the underlying storage mechanism (SQL, NoSQL, API, Memory),
allowing the ObjectQL engine to execute standardized CRUD and Aggregation operations
regardless of where the data resides.
The Data Engine acts as the "Driver" layer in the Hexagonal Architecture.
Source: packages/spec/src/data/data-engine.zod.ts
import { BaseEngineOptionsSchema, DataEngineAggregateOptionsSchema, DataEngineAggregateRequestSchema, DataEngineCountOptionsSchema, DataEngineCountRequestSchema, DataEngineDeleteOptionsSchema, DataEngineDeleteRequestSchema, DataEngineExecuteRequestSchema, DataEngineFilterSchema, DataEngineFindOneRequestSchema, DataEngineFindRequestSchema, DataEngineInsertOptionsSchema, DataEngineInsertRequestSchema, DataEngineQueryOptionsSchema, DataEngineRequestSchema, DataEngineSortSchema, DataEngineUpdateOptionsSchema, DataEngineUpdateRequestSchema, DataEngineVectorFindRequestSchema, DroppedFieldsEventSchema, EngineAggregateOptionsSchema, EngineCountOptionsSchema, EngineDeleteOptionsSchema, EngineQueryOptionsSchema, EngineUpdateOptionsSchema } from '@objectstack/spec/data' ; import type { BaseEngineOptions, DataEngineAggregateOptions, DataEngineCountOptions, DataEngineDeleteOptions, DataEngineExecuteRequest, DataEngineFilter, DataEngineInsertOptions, DataEngineInsertRequest, DataEngineQueryOptions, DataEngineRequest, DataEngineSort, DataEngineUpdateOptions, DataEngineVectorFindRequest, DroppedFieldsEvent, EngineAggregateOptions, EngineCountOptions, EngineDeleteOptions, EngineQueryOptions, EngineUpdateOptions } from '@objectstack/spec/data' ; // Validate data const result = BaseEngineOptionsSchema. parse (data); Property Type Required Description context { userId?: string; actor?: string; attributedUserId?: string; email?: string; … }optional
Options for DataEngine.aggregate operations
Property Type Required Description context { userId?: string; actor?: string; attributedUserId?: string; email?: string; … }optional filter Record<string, any> | anyoptional Data Engine query filter conditions groupBy string[]optional aggregations { field: string; method: Enum<'count' | 'sum' | 'avg' | 'min' | 'max' | 'count_distinct'>; alias?: string }[]optional
Property Type Required Description method 'aggregate'✅ object string✅ query { context?: object; where?: Record<string, any> | any; groupBy?: (string | object)[]; aggregations?: object[]; … }✅
Options for DataEngine.count operations
Property Type Required Description context { userId?: string; actor?: string; attributedUserId?: string; email?: string; … }optional filter Record<string, any> | anyoptional Data Engine query filter conditions
Property Type Required Description method 'count'✅ object string✅ query { context?: object; where?: Record<string, any> | any; filter?: Record<string, any> | any }optional
Options for DataEngine.delete operations
Property Type Required Description context { userId?: string; actor?: string; attributedUserId?: string; email?: string; … }optional filter Record<string, any> | anyoptional Data Engine query filter conditions multi booleanoptional
Property Type Required Description method 'delete'✅ object string✅ id string | numberoptional ID for single delete, or use where in options options { context?: object; where?: Record<string, any> | any; multi?: boolean; filter?: Record<string, any> | any }optional
Property Type Required Description method 'execute'✅ command any✅ options Record<string, any>optional
Data Engine query filter conditions
This schema accepts one of the following structures:
Type: Record<string, any>
Reference: any
Property Type Required Description method 'findOne'✅ object string✅ query { context?: object; where?: Record<string, any> | any; fields?: string[]; orderBy?: object[]; … }optional
Property Type Required Description method 'find'✅ object string✅ query { context?: object; where?: Record<string, any> | any; fields?: string[]; orderBy?: object[]; … }optional
Options for DataEngine.insert operations
Property Type Required Description context { userId?: string; actor?: string; attributedUserId?: string; email?: string; … }optional returning booleanoptional
Property Type Required Description method 'insert'✅ object string✅ data Record<string, any> | Record<string, any>[]✅ options { context?: object; returning?: boolean }optional Options for DataEngine.insert operations
Query options for IDataEngine.find() operations
Property Type Required Description context { userId?: string; actor?: string; attributedUserId?: string; email?: string; … }optional filter Record<string, any> | anyoptional Data Engine query filter conditions select string[]optional sort Record<string, Enum<'asc' | 'desc'>> | Record<string, 1 | -1> | { field: string; order: Enum<'asc' | 'desc'> }[]optional Sort order definition limit integeroptional skip integeroptional top integeroptional populate string[]optional
Virtual ObjectQL Request Protocol
This schema accepts one of the following structures:
Property Type Required Description method 'find'✅ object string✅ query { context?: object; where?: Record<string, any> | any; fields?: string[]; orderBy?: object[]; … }optional
Property Type Required Description method 'findOne'✅ object string✅ query { context?: object; where?: Record<string, any> | any; fields?: string[]; orderBy?: object[]; … }optional
Property Type Required Description method 'insert'✅ object string✅ data Record<string, any> | Record<string, any>[]✅ options { context?: object; returning?: boolean }optional Options for DataEngine.insert operations
Property Type Required Description method 'update'✅ object string✅ data Record<string, any>✅ id string | numberoptional ID for single update, or use where in options options { context?: object; where?: Record<string, any> | any; multi?: boolean; returning?: boolean; … }optional
Property Type Required Description method 'delete'✅ object string✅ id string | numberoptional ID for single delete, or use where in options options { context?: object; where?: Record<string, any> | any; multi?: boolean; filter?: Record<string, any> | any }optional
Property Type Required Description method 'count'✅ object string✅ query { context?: object; where?: Record<string, any> | any; filter?: Record<string, any> | any }optional
Property Type Required Description method 'aggregate'✅ object string✅ query { context?: object; where?: Record<string, any> | any; groupBy?: (string | object)[]; aggregations?: object[]; … }✅
Property Type Required Description method 'execute'✅ command any✅ options Record<string, any>optional
Property Type Required Description method 'vectorFind'✅ object string✅ vector number[]✅ where Record<string, any> | anyoptional fields string[]optional limit integeroptional threshold numberoptional
Sort order definition
This schema accepts one of the following structures:
Type: Record<string, Enum<'asc' | 'desc'>>
Type: Record<string, 1 | -1>
Type: { field: string; order: Enum<'asc' | 'desc'> }[]
Options for DataEngine.update operations
Property Type Required Description context { userId?: string; actor?: string; attributedUserId?: string; email?: string; … }optional filter Record<string, any> | anyoptional Data Engine query filter conditions upsert neveroptional [REMOVED] update.options.upsert was removed in @objectstack/spec 17 (#8057, ADR-0049) — it was declared and allowlisted but never implemented: no engine or driver path ever read it, so { upsert: true } was accepted and silently dropped and the update stayed a plain update. Delete the key. Express create-if-absent explicitly: a by-id update whose id names no row throws RECORD_NOT_FOUND (#7867's not-found gate) rather than inserting, so read the row first (findOne) and call insert or update on what you find. A first-class upsert, if ever built, must reconcile with that gate by design rather than through this silent flag. multi booleanoptional returning booleanoptional
Property Type Required Description method 'update'✅ object string✅ data Record<string, any>✅ id string | numberoptional ID for single update, or use where in options options { context?: object; where?: Record<string, any> | any; multi?: boolean; returning?: boolean; … }optional
Property Type Required Description method 'vectorFind'✅ object string✅ vector number[]✅ where Record<string, any> | anyoptional fields string[]optional limit integeroptional threshold numberoptional
A write-path strip event: caller-supplied fields legally dropped from the payload (#3407)
Property Type Required Description object string✅ Object the write targeted (resolved object name) fields string[]✅ Caller-supplied field names the engine removed from the write payload reason Enum<'readonly' | 'readonly_when' | 'primary_key'>✅ Why the fields were dropped: static readonly (#2948), a TRUE readonlyWhen predicate (#3042), or the primary-key strip of a payload id the engine ruled is not an identifier (#6437)
QueryAST-aligned options for DataEngine.aggregate operations
Property Type Required Description context { userId?: string; actor?: string; attributedUserId?: string; email?: string; … }optional where Record<string, any> | anyoptional groupBy (string | { field: string; dateGranularity?: Enum<'day' | 'week' | 'month' | 'quarter' | 'year'>; alias?: string })[]optional GROUP BY targets (strings or {field, dateGranularity?} objects for date bucketing) aggregations { function: Enum<'count' | 'sum' | 'avg' | 'min' | 'max' | 'count_distinct'>; field?: string; alias: string; filter?: any }[]optional having anyoptional HAVING — filter over the aggregated rows (aggregation aliases + groupBy projections); applied engine-side after aggregation timezone stringoptional
QueryAST-aligned options for DataEngine.count operations
Property Type Required Description context { userId?: string; actor?: string; attributedUserId?: string; email?: string; … }optional where Record<string, any> | anyoptional
QueryAST-aligned options for DataEngine.delete operations
Property Type Required Description context { userId?: string; actor?: string; attributedUserId?: string; email?: string; … }optional where Record<string, any> | anyoptional multi booleanoptional
QueryAST-aligned query options for IDataEngine.find() operations
Property Type Required Description context { userId?: string; actor?: string; attributedUserId?: string; email?: string; … }optional where Record<string, any> | anyoptional fields string[]optional orderBy { field: string; order: Enum<'asc' | 'desc'> }[]optional limit numberoptional offset numberoptional top numberoptional cursor neveroptional [REMOVED] query.cursor was removed in @objectstack/spec 17 (#4286, ADR-0049) — no driver ever implemented keyset pagination, so the cursor was accepted and ignored and every page came back identical (a caller looping "until hasMore is false" never terminates). Delete the key; QueryBuilder.cursor() was removed with it. Express the keyset as an ordinary where predicate on your sort key — where: { created_at: { $gt: last.created_at } } with the matching orderBy — which every driver executes with canonicalised comparands. A first-class cursor, if ever built, will be a response-minted opaque token, not this caller-built record. search string | { query: string; fields?: string[]; fuzzy: boolean; operator: Enum<'and' | 'or'>; … }optional searchFields string[]optional expand Record<string, { object: string; fields?: string[]; where?: any; search?: string | object; … }>optional distinct neveroptional [REMOVED] query.distinct was removed in @objectstack/spec 17 (#4286, ADR-0049 / ADR-0078) — no driver ever rendered SELECT DISTINCT; the flag's only observable effect was MIS-WIRED: the REST list path treated a distinct query as not countable and silently degraded total/hasMore to a page-local estimate while still returning duplicate rows. Delete the key; QueryBuilder.distinct() was removed with it, and the count suppression is gone (total is truthful again). For unique values of one column use the SQL/memory drivers' distinct(object, field) door; for unique combinations, groupBy; for a deduplicated count, the count_distinct aggregation.
QueryAST-aligned options for DataEngine.update operations
Property Type Required Description context { userId?: string; actor?: string; attributedUserId?: string; email?: string; … }optional where Record<string, any> | anyoptional upsert neveroptional [REMOVED] update.options.upsert was removed in @objectstack/spec 17 (#8057, ADR-0049) — it was declared and allowlisted but never implemented: no engine or driver path ever read it, so { upsert: true } was accepted and silently dropped and the update stayed a plain update. Delete the key. Express create-if-absent explicitly: a by-id update whose id names no row throws RECORD_NOT_FOUND (#7867's not-found gate) rather than inserting, so read the row first (findOne) and call insert or update on what you find. A first-class upsert, if ever built, must reconcile with that gate by design rather than through this silent flag. multi booleanoptional returning booleanoptional
ObjectStack