Feature #700

Please add BLOB streaming functions to WT::DBO

Added by Anonymous over 10 years ago. Updated over 10 years ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:


for Postgresql :

- lo_create() : to create a large object

- lo_open() : to open a large object

- lo_read() : to read from a large object

- lo_write() : to write data to large object

- lo_close() : to close a large object

  • lo_unlink() : to free resources in large object table


- sqlite3_blob_open()

- sqlite3_blob_read ()

- sqlite3_blob_write()

  • sqlite3_blob_close()

Updated by Koen Deforche over 10 years ago

  • Status changed from New to Feedback


This should be relatively straightforward, I believe, but alot more work than other type mappings. The C type on which this maps should probably be a specialized c iostream. That means that we need to implement a custom streambuf.

It's not a priority to us at the moment, but we are willing to help anyone adding this !




Updated by Koen Deforche over 10 years ago

2011/2/9 Dmitriy Igrishin <>:

>> I guess this corresponds to this feature request:


Indeed, but please note, that any operation on large object (LOB) must be

inside transaction. So, I guess that specialized std::iostream must

encapsulate connection object and OID of LOB and open a transaction

in the constructor and close it in the destuctor ("resource acquisition is

initialization" technique).

I was thinking of requiring that a user already has a transaction,

since that is the assumption in other parts of dbo. In general, you

want to have a transaction before manipulating (reading/writing) a

database object since there is all sorts of lazy loading.

Also, a database object currently cannot be shared between sessions,

and is not thread safe. If you want to use the database object from

inside a resource, you need to properly guard it.

Now, since resources are reentrant by default, that does not work well

out of the box with database objects (and blobs). Therefore, I propose

that for each request to a resource that accesses a blob:

1) creates a Dbo::Session

2) it opens a Dbo::Transaction (which in turn grabs a connection from the pool)

3) locks the database object (with a custom boost::mutex)

3.a) gets a std::istream object to read from the blob

4) handles the request (as you describe)

5) closes the transaction

6) closes the session.

Currently creating a Dbo::Session is potentially expensive, but we

could optimize that too by using a session pool. Steps 1 and 6 then

become "get" and "return" a session from the session pool.

Yes, of course, std::istream should not be shared between threads.

It must be constructed locally inside handleRequest(). In case of LOBs,

each call of handleRequest() will get a connection (probably from pool),

construct LobStream then will perform the same algorithm as in

current WFileResource.


>> In any case, I believe it diserves a feature request and perhaps

>> WFileResource could be refactored to be a specializiatino of a

>> WStreamResource.

So, maybe we need to add WStreamResource with a useful protected

method (e.g. doHandleRequest()) which operates with given std::istream

and called from handleRequest() (just generalization of current

WFileResource::handleRequest()) ?

Because the algorithm is common.

Also available in: Atom PDF