vectorstores: add pgvector

[Abirdcfly]

PR Checklist

• Read the Contributing documentation.
• Read the Code of conduct documentation.
• Name your Pull Request title clearly, concisely, and prefixed with the name of the primarily affected package you changed according to Good commit messages (such as memory: add interfaces for X, Y or util: add whizzbang helpers).
• Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
• Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. Fixes #123).
• Describes the source of new concepts.
• References existing implementations as appropriate.
• Contains test coverage for new functions.
• Passes all golangci-lint checks.

[tmc]

rather than string-assemble this can you use arguments to Exec (throughout) -- this protects against sql injection attacks.

[Abirdcfly]

IMO, according to https://www.postgresql.org/docs/current/xfunc-sql.html#XFUNC-SQL-FUNCTION-ARGUMENTS: SQL function arguments can only be used as data values, not as identifiers. replacing the table name with $1 does not seem to work.

[eliben]

I think Travis maybe means that you shouldn't Sprintf into package sql calls like Exec, but rather use the args of each method, e.g. look at the example at https://pkg.go.dev/database/sql#example-Conn.ExecContext

result, err := conn.ExecContext(ctx,UPDATE balances SET balance = balance + 10 WHERE user_id = ?;, id)

See how id is passed into the SQL query?

[tmc]

Yes that's what I meant.

[Abirdcfly]

I think I may be getting it, but it seems difficult to do this if we wants to keep the table name as a parameter create table ? or create table $1 will both report an error, and I have to use the Sprint function and do Sanitize() when the user passes in the table name.

Or, we can fix the table name to be a constant and unchangeable.

Everywhere else, $1 is used to replace the parameter.

[Abirdcfly]

kindly ping @eliben @tmc

[tmc]

Please add/include an example showing usage.

[Abirdcfly]

Please add/include an example showing usage.

Should I refer to the vectorstores/pgvector/pgvector_test.go file in examples dir to write an example of storing vectors using openai llm and pgvector? 🤔

[eliben]

I think Travis maybe means that you shouldn't Sprintf into package sql calls like Exec, but rather use the args of each method, e.g. look at the example at https://pkg.go.dev/database/sql#example-Conn.ExecContext

result, err := conn.ExecContext(ctx,UPDATE balances SET balance = balance + 10 WHERE user_id = ?;, id)

See how id is passed into the SQL query?

[eliben]

I think my review comments have been addressed

On to @tmc to approve and merge if he's happy with the PR

[tmc]

LGTM!

[tmc]

Please add/include an example showing usage.

Should I refer to the vectorstores/pgvector/pgvector_test.go file in examples dir to write an example of storing vectors using openai llm and pgvector? 🤔

That would work, I'd love to see more ollama examples but you can you openai, up to you.