Feature #700
Please add BLOB streaming functions to WT::DBO
0%
Description
for Postgresql : ¶
- lo_create() : to create a large object
- lo_open() : to open a large object
- lo_read() : to read from a large object
- lo_write() : to write data to large object
- lo_close() : to close a large object
- lo_unlink() : to free resources in large object table
For SQLITE3 : ¶
- sqlite3_blob_open()
- sqlite3_blob_read ()
- sqlite3_blob_write()
- sqlite3_blob_close()
Updated by Koen Deforche over 12 years ago
- Status changed from New to Feedback
Hey,
This should be relatively straightforward, I believe, but alot more work than other type mappings. The C type on which this maps should probably be a specialized c iostream. That means that we need to implement a custom streambuf.
It's not a priority to us at the moment, but we are willing to help anyone adding this !
Regards,
koen
Updated by Koen Deforche about 12 years ago
2011/2/9 Dmitriy Igrishin <dmitigr@gmail.com>:
>> I guess this corresponds to this feature request:
>> http://redmine.emweb.be/issues/700
Indeed, but please note, that any operation on large object (LOB) must be
inside transaction. So, I guess that specialized std::iostream must
encapsulate connection object and OID of LOB and open a transaction
in the constructor and close it in the destuctor ("resource acquisition is
initialization" technique).
I was thinking of requiring that a user already has a transaction,
since that is the assumption in other parts of dbo. In general, you
want to have a transaction before manipulating (reading/writing) a
database object since there is all sorts of lazy loading.
Also, a database object currently cannot be shared between sessions,
and is not thread safe. If you want to use the database object from
inside a resource, you need to properly guard it.
Now, since resources are reentrant by default, that does not work well
out of the box with database objects (and blobs). Therefore, I propose
that for each request to a resource that accesses a blob:
1) creates a Dbo::Session
2) it opens a Dbo::Transaction (which in turn grabs a connection from the pool)
3) locks the database object (with a custom boost::mutex)
3.a) gets a std::istream object to read from the blob
4) handles the request (as you describe)
5) closes the transaction
6) closes the session.
Currently creating a Dbo::Session is potentially expensive, but we
could optimize that too by using a session pool. Steps 1 and 6 then
become "get" and "return" a session from the session pool.
Yes, of course, std::istream should not be shared between threads.
It must be constructed locally inside handleRequest(). In case of LOBs,
each call of handleRequest() will get a connection (probably from pool),
construct LobStream then will perform the same algorithm as in
current WFileResource.
>>
>> In any case, I believe it diserves a feature request and perhaps
>> WFileResource could be refactored to be a specializiatino of a
>> WStreamResource.So, maybe we need to add WStreamResource with a useful protected
method (e.g. doHandleRequest()) which operates with given std::istream
and called from handleRequest() (just generalization of current
WFileResource::handleRequest()) ?
Because the algorithm is common.