S-SQL reference manual

This is the reference manual for the S-SQL component of the postmodern library.

S-SQL provides a lispy syntax for SQL queries, and knows how to convert various lisp types to their textual SQL representation. It takes care to do as much of the work as possible at compile-time, so that at runtime a string concatenation is all that is needed to produce the final SQL query.

Interface
SQL Types
SQL Syntax
Symbol-index

Interface

macro sql (form)
→ string

Convert the given form (a list starting with a keyword) to an SQL query string at compile time, according to the rules described here.

function sql-compile (form)
→ string

This is the run-time variant of the sql macro. It converts the given list to an SQL query, with the same rules except that symbols in this list do not have to be quoted to be interpreted as identifiers.

function sql-template (form)

In cases where you do need to build the query at run time, yet you do not want to re-compile it all the time, this function can be used to compile it once and store the result. It takes an S-SQL form, which may contain $$ placeholder symbols, and returns a function that takes one argument for every $$. When called, this returned function produces an SQL string in which the placeholders have been replaced by the values of the arguments.

macro enable-s-sql-syntax (&optional (char #\Q))

Modifies the current readtable to add a #Q syntax that is read as (sql ...). The character to use can be overridden by passing an argument.

function sql-escape-string (string)
→ string

Escapes a string for inclusion in a PostgreSQL query.

method sql-escape (value)
→ string

A generalisation of sql-escape-string. Looks at the type of the value passed, and properly writes it out it for inclusion in an SQL query. Symbols will be converted to SQL names.

variable *standard-sql-strings*

Used to configure whether S-SQL will use standard SQL strings (just replace #\' with ''), or backslash-style escaping. Setting this to NIL is always safe, but when the server is configured to allow standard strings (compile-time parameter 'standard_conforming_strings' is 'on', which will become the default in future versions of PostgreSQL), the noise in queries can be reduced by setting this to T.

variable *escape-sql-names-p*

Determines whether double quotes are added around column, table, and function names in queries. May be T, in which case every name is escaped, NIL, in which case none is, or :auto, which causes only reserved words to be escaped.. The default value is :auto. Be careful when binding this with let and such ― since a lot of SQL compilation tends to happen at compile-time, the result might not be what you expect.

function sql-type-name (type)
→ string

Create the SQL equivalent of the given Lisp type, if one is known. See types.

function to-sql-name (name &optional (escape-p *escape-sql-names-p*))
→ string

Convert a symbol or string to a name that can be used as an SQL identifier by converting all non-alphanumeric characters to underscores. Also lowercases the name to make queries look a bit less hideous. When a second argument is given, this overrides the current value of *escape-sql-names-p*.

function from-sql-name (string)
→ keyword

Convert a string that represents an SQL identifier to a keyword by uppercasing it and converting the underscores to dashes.

macro register-sql-operators (arity &rest names)

Define simple SQL operators. Arity is one of :unary (like 'not'), :unary-postfix (the operator comes after the operand), :n-ary (like '+': the operator falls away when there is only one operand), :2+-ary (like '=', which is meaningless for one operand), or :n-or-unary (like '-', where the operator is kept in the unary case). After the arity may follow any number of operators, either just a keyword, in which case the downcased symbol name is used as the SQL operator, or a two-element list containing a keyword and a name string.

SQL Types

S-SQL knows the SQL equivalents to a number of Lisp types, and defines some extra types that can be used to denote other SQL types. The following table shows the correspondence:

Lisp type	SQL type
smallint	smallint
integer	integer
bigint	bigint
(numeric X Y)	numeric(X, Y)
float, real	real
double-float, double-precision	double-precision
string, text	text
(string X)	char(X)
(varchar X)	varchar(X)
boolean	boolean
bytea	bytea
date	date
timestamp	timestamp
interval	interval

type db-null

This is a type of which only the keyword :null is a member. It is used to represent NULL values from the database.

SQL Syntax

An S-SQL form is converted to a query through the following rules:

Lists starting with a keyword are operators. They are expanded as described below if they are known, otherwise they are expanded in the standard way: operator(arguments, ...)
Quoted symbols or keywords are interpreted as names of columns or tables, and converted to strings with to-sql-name.
Anything else is evaluated and the resulting Lisp value is converted to its textual SQL representation (or an error is raised when there is no rule for converting objects of this type). Self-quoting atoms may be converted to strings at compile-time.

The following operators are defined:

sql-op :+, :*, :%, :&, :|, :||, :and, :or, :=, :/, :!=, :<, :>, :<=, :>=, :^, :union, :union-all, :intersect, :intersect-all, :except, :except-all (&rest args)

These are expanded as infix operators. When meaningful, they allow more than two arguments. :- can also be used as a unary operator to negate a value. Note that the arguments to :union, :union-all, :intersect, and :except should be queries (:select forms).

sql-op :~, :not (arg)

Unary operators for bitwise and logical negation.

sql-op :~, :~*, :!~, :!~* (string pattern)

Regular expression matching operators. The exclamation mark means 'does not match', the asteriks makes the match case-insensitive.

sql-op :like, :ilike (string pattern)

Simple SQL string matching operators (:ilike is case-insensitive).

sql-op :desc (column)

Used to invert the meaning of an operator in an :order-by clause.

sql-op :nulls-first, :nulls-last (column)

Used to determine where :null values appear in an :order-by clause.

sql-op :as (form name &rest fields)

Assigns a name to a column or table in a :select form. When fields are given, they are added after the name, in parentheses. For example, (:as 'table1 't1 'foo 'bar) becomes table1 AS t1(foo, bar). When you need to specify types for the fields, you can do something like (:as 'table2 't2 ('foo integer)). Note that names are quoted, types are not (when using sql-compile or sql-template, you can leave out the quotes entirely).

sql-op :exists (query)

The EXISTS operator. Takes a query as an argument, and returns true or false depending on whether that query returns any rows.

sql-op :is-null (arg)

Test whether a value is null.

sql-op :in (value set)

Test whether a value is in a set of values.

sql-op :not-in (value set)

Inverse of the above.

sql-op :set (&rest elements)

Denote a set of values. This one has two interfaces. When the elements are known at compile-time, they can be given as multiple arguments to the operator. When they are not, a single argument that evaulates to a list should be used.

sql-op :[] (form subscript)

Dereference an array value.

sql-op :extract (unit form)

Extract a field from a date/time value. For example, (:extract :month (:now)).

sql-op :case (&rest clauses)

A conditional expression. Clauses should take the form (test value).

sql-op :between (n start end)

Test whether a value lies between two other values.

sql-op :between-symmetric (n start end)

Works like :between, except that the start value is not required to be less than the end value.

sql-op :dot (&rest names)

Can be used to combine multiple names into a name of the form A.B to refer to a column in a table, or a table in a schema. Note that you calso just use a symbol with a dot in it.

sql-op :type (form type)

Add a type declaration to a value, as in in "4.3::real". The second argument is not evaluated normally, but put through sql-type-name to get a type identifier.

sql-op :raw (string)

Insert a string as-is into the query. This can be useful for doing things that the syntax does not support, or to re-use parts of a query across multiple queries:

(let* ((test (sql (:and (:= 'foo 22) (:not-null 'bar))))
       (rows (query (:select '* :from 'baz :where (:raw test)))))
  (query (:delete-from 'baz :where (:raw test)))
  (do-stuff rows))

sql-op :select (&rest args)

Creates a select query. The arguments are split on the keywords found among them. The group of arguments immediately after :select is interpreted as the expressions that should be selected. After this an optional :distinct may follow, which will cause the query to only select distinct rows, or alternatively :distinct-on followed by a group of row names. Next comes the optional keyword :from, followed by at least one table name and then any numer of join statements. Join statements start with one of :left-join, :right-join, :inner-join, :outer-join or :cross-join, then a table name or subquery, then the keyword :on, if applicable, and then a form. A join can be preceded by :natural (leaving off the :on clause) to use a natural join. After the joins an optional :where followed by a single form may occur. And finally :group-by and :having can optionally be specified. The first takes any number of arguments, and the second only one. An example:

(:select (:+ 'field-1 100) 'field-5
   :from (:as 'my-table 'x)
   :left-join 'your-table :on (:= 'x.field-2 'your-table.field-1)
   :where (:not-null 'a.field-3))

sql-op :limit (query amount &optional offset)

In S-SQL limit is not part of the select operator, but an extra operator that is applied to a query (this works out better when limiting the union or intersection of multiple queries, same for sorting). It limits the number of results to the amount given as the second argument, and optionally offsets the result by the amount given as the third argument.

sql-op :order-by (query &rest exprs)

Order the results of a query by the given expressions. See :desc for when you want to invert an ordering.

sql-op :function (name (&rest arg-types) return-type stability body)

Create a stored procedure. The argument and return types are interpreted as type names and not evaluated. Stability should be one of :immutable, :stable, or :volatile (see the Postgres manual). For example, a function that gets foobars by id:

(:function 'get-foobar (integer) foobar :stable (:select '* :from 'foobar :where (:= 'id '$1)))

sql-op :insert-into (table &rest rest)

Insert a row into a table. When the second argument is :set, the other arguments should be alternating field names and values, otherwise it should be a :select form that will produce the values to be inserted. Example:

(:insert-into 'my-table :set 'field-1 42 'field-2 "foobar")

It is possible to add :returning, followed by a list of field names or expressions, at the end of the :insert-into form. This will cause the query to return the values of these expressions as a single row.

sql-op :update (table &rest rest)

Update values in a table. After the table name there should follow the keyword :set and any number of alternating field names and values, like for :insert-into, and then an optional :where keyword followed by the condition, and :returning keyword followed by a list of field names or expressions indicating values to be returned as query result.

sql-op :delete-from (table &rest rest)

Delete rows from the named table. Can be given a :where argument followed by a condition, and a :returning argument, followed by one or more expressions that should be returned for every deleted row.

sql-op :create-table (name (&rest columns) &rest options)

Create a new table. After the table name a list of column definitions follows, which are lists that start with a name, followed by one or more of the following keyword arguments:

:type: This one is required. It specifies the type of the column. Use a type like (or db-null integer) to specify a column that may have NULL values.
:default: Provides a default value for the field.
:unique: If this argument is non-nil, the values of the column must be unique.
:primary-key: When non-nil, the column is a primary key of the table.
:check: Adds a constraint to this column. The value provided for this argument must be an S-SQL expression that returns a boolean value. It can refer to other columns in the table if needed.
:references: Adds a foreign key constraint to this table. The argument provided must be a list of the form (target &optional on-delete on-update). When target is a symbol, it names the table to whose primary key this constraint refers. When it is a list, its first element is the table, and its second element the column within that table that the key refers to. on-delete and on-update can be used to specify the actions that must be taken when the row that this key refers to is deleted or changed. Allowed values are :restrict, :set-null, :set-default, :cascade, and :no-action.

After the list of columns, zero or more extra options (table constraints) can be specified. These are lists starting with one of the following keywords:

:check: Adds a constraint to the table. Takes a single S-SQL expression that produces a boolean as its argument.
:primary-key: Specifies a primary key for the table. The arguments to this option are the names of the columns that this key consists of.
:unique: Adds a unique constraint to a group of columns. Again, the arguments are a list of symbols that indicate the relevant columns.
:foreign-key: Create a foreign key. The arguments should have the form (columns target &optional on-delete on-update), where columns is a list of columns that are used by this key, while the rest of the arguments have the same meaning as they have in the :references option for columns.

Every list can start with :constraint name to create a specifically named constraint.

Note that, unlike most other operators, :create-table expects most of its arguments to be unquoted symbols. The exception to this is the value of :check constraints: These must be normal S-SQL expressions, which means that any column names they contain should be quoted. When programatically generating table definitions, sql-compile is usually more practical than the sql macro.

Here is an example of a :create-table form:

(:create-table enemy
  ((name :type string :primary-key t)
   (age :type integer)
   (address :type (or db-null string) :references (important-addresses :cascade :cascade))
   (fatal-weakness :type text :default "None")
   (identifying-color :type (string 20) :unique t)
  (:foreign-key (identifying-color) (colors name))
  (:constraint enemy-age-check :check (:> 'age 12)))

sql-op:alter-table (name action &rest args)

Alters named table. Currently, only adding and dropping constraints are supported. Meaning of args depends on action:

:add: Adds constraint to table. args should be a constraint in the same form as for :create-table.
:add-column: Adds column to table. args should be a column in the same form as for :create-table.
:drop-constraint: Drops constraint. First of args should name a constraint to be dropped; second, optional argument specifies behaviour regarding objects dependent on the constraint and it may equal :cascade or :restrict.

sql-op :drop-table (name)

Drops the named table. You may optionally pass :if-exists before the name to suppress the error message.

sql-op :create-index (name &rest args)

Create an index on a table. After the name of the index the keyword :on should follow, with the table name after it. Then the keyword :fields, followed by one or more column names. Optionally, a :where clause with a condition can be added at the end to make a partial index.

sql-op :create-unique-index (name &rest args)

Works like :create-index, except that the index created is unique.

sql-op :drop-index (name)

Drop an index. Takes an :if-exists argument like :drop-table.

sql-op :create-sequence (name &key increment min-value max-value start cache cycle)

Create a sequence with the given name. The rest of the arguments control the way the sequence selects values.

sql-op :drop-sequence (name)

Drop a sequence. You may pass :if-exists as an extra first argument.

sql-op :create-view (name query)

Create a view from an S-SQL-style query.

sql-op :drop-view (name)

Drop a view. Takes optional :if-exists argument.

Symbol-index

:+
:-
:*
:&
:|
:||
:=
:/
:!=
:<
:>
:<=
:>=
:^
:~
:!~
:!~*
:~*
:[]
$$
:and
:as
:between
bytea
bigint
:case
:create-index
:create-sequence
:create-table
:create-unique-index
db-null
:delete-from
:desc
:dot
double-precision
:drop-index
:drop-sequence
:drop-table
:drop-view
enable-s-sql-syntax
*escape-sql-names-p*
:except
:exists
:extract
from-sql-name
:function
:ilike
:in
:insert-into
:intersect
:is-null
:like
:limit
:not
:not-in
numeric
:nulls-first
:nulls-last
:or
:order-by
:raw
real
:register-sql-operators
*standard-sql-strings*
:select
:set
smallint
sql
sql-compile
sql-escape
sql-escape-string
sql-template
sql-type-name
text
to-sql-name
:type
:union
:union-all
:update
varchar

S-SQL reference manual

Contents

Interface

SQL Types

SQL Syntax

Symbol-index