- Timestamp:
- 6 Nov 2011, 23:36:21 (13 years ago)
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
main/waeup.sirp/branches/ulif-extimgstore/src/waeup/sirp/imagestorage.py
r7002 r7010 21 21 ## 22 22 """A storage for image files. 23 24 A few words about storing files with ``waeup.sirp``. The need for this 25 feature arised initially from the need to store passport files for 26 applicants and students. These files are dynamic (can be changed 27 anytime), mean a lot of traffic and cost a lot of memory/disk space. 28 29 **Design Basics** 30 31 While one *can* store images and similar 'large binary objects' aka 32 blobs in the ZODB, this approach quickly becomes cumbersome and 33 difficult to understand. The worst approach here would be to store 34 images as regular byte-stream objects. ZODB supports this but 35 obviously access is slow (data must be looked up in the one 36 ``Data.fs`` file, each file has to be sent to the ZEO server and back, 37 etc.). 38 39 A bit less worse is the approach to store images in the ZODB but as 40 Blobs. ZODB supports storing blobs in separate files in order to 41 accelerate lookup/retrieval of these files. The files, however, have 42 to be sent to the ZEO server (and back on lookups) which means a 43 bottleneck and will easily result in an increased number of 44 ``ConflictErrors`` even on simple reads. 45 46 The advantage of both ZODB-geared approaches is, of course, complete 47 database consistency. ZODB will guarantee that your files are 48 available under some object name and can be handled as any other 49 Python object. 50 51 Another approach is to leave the ZODB behind and to store images and 52 other files in filesystem directly. This is faster (no ZEO contacts, 53 etc.), reduces probability of `ConflictErrors`, keeps the ZODB 54 smaller, and enables direct access (over filesystem) to the 55 files. Furthermore steps might be better understandable for 56 third-party developers. We opted for this last option. 57 58 **External File Store** 59 60 Our implementation for storing-files-API is defined in 61 :class:`ExtFileStore`. An instance of this file storage (which is also 62 able to store non-image files) is available at runtime as a global 63 utility implementing :class:`waeup.sirp.interfaces.IExtFileStore`. 64 65 The main task of this central component is to maintain a filesystem 66 root path for all files to be stored. It also provides methods to 67 store/get files under certain file ids which identify certain files 68 locally. 69 70 So, to store a file away, you can do something like this: 71 72 >>> from StringIO import StringIO 73 >>> from zope.component import getUtility 74 >>> from waeup.sirp.interfaces import IExtFileStore 75 >>> store = getUtility(IExtFileStore) 76 >>> store.createFile('myfile.txt', StringIO('some file content')) 77 78 All you need is a filename and the file-like object containing the 79 real file data. 80 81 This will store the file somewhere (you shouldn't make too much 82 assumptions about the real filesystem path here). 83 84 Later, we can get the file back like this: 85 86 >>> store.getFile('myfile.txt') 87 <open file ...> 88 89 What we get back is a file or file-like object already opened for 90 reading: 91 92 >>> store.getFile('myfile.txt').read() 93 'some file content' 94 95 **Handlers: Special Places for Special Files** 96 97 The file store supports special handling for certain files. For 98 example we want applicant images to be stored in a different directory 99 than student images, etc. Because the file store cannot know all 100 details about these special tratment of certain files, it looks up 101 helpers (handlers) to provide the information it needs for really 102 storing the files at the correct location. 103 104 That a file stored in filestore needs special handling can be 105 indicated by special filenames. These filenames start with a marker like 106 this:: 107 108 __<MARKER-STRING>__real-filename.jpg 109 110 Please note the double underscores before and after the marker 111 string. They indicate that all in between is a marker. 112 113 If you store a file in file store with such a filename (we call this a 114 `file_id` to distuingish it from real world filenames), the file store 115 will look up a handler for ``<MARKER-STRING>`` and pass it the file to 116 store. The handler then will return the internal path to store the 117 file and possibly do additional things as well like validating the 118 file or similar. 119 120 Examples for such a file store handler can be found in the 121 :mod:`waeup.sirp.applicants.applicant` module. Please see also the 122 :class:`DefaultFileStoreHandler` class below for more details. 123 124 The file store looks up handlers by utility lookups: it looks for a 125 named utiliy providing 126 :class:`waeup.sirp.interfaces.IFileStoreHandler` and named like the 127 marker string (without leading/trailing underscores) in lower 128 case. For example if the file id would be 129 130 ``__IMG_USER__manfred.jpg`` 131 132 then the looked up utility should be registered under name 133 134 ``img_user`` 135 136 and provide :class:`waeup.sirp.interfaces.IFileStoreHandler`. If no 137 such utility can be found, a default handler is used instead 138 (see :class:`DefaultFileStoreHandler`). 139 140 **Context Adapters: Knowing Your Family** 141 142 Often the internal filename or file id of a file depends on a 143 context. For example when we store passport photographs of applicants, 144 then each image belongs to a certain applicant instance. It is not 145 difficult to maintain such a connection manually: Say every applicant 146 had an id, then we could put this id into the filename as well and 147 would build the filename to store/get the connected file by using that 148 filename. You then would create filenames of a format like this:: 149 150 __<MARKER-STRING>__applicant0001.jpg 151 152 where ``applicant0001`` would tell exactly which applicant you can see 153 on the photograph. You notice that the internal file id might have 154 nothing to do with once uploaded filenames. The id above could have 155 been uploaded with filename ``manfred.jpg`` but with the new file id 156 we are able to find the file again later. 157 158 Unfortunately it might soon get boring or cumbersome to retype this 159 building of filenames for a certain type of context, especially if 160 your filenames take more of the context into account than only a 161 simple id. 162 163 Therefore you can define filename building for a context as an adapter 164 that then could be looked up by other components simply by doing 165 something like: 166 167 >>> from waeup.sirp.interfaces import IFileStoreNameChooser 168 >>> file_id = IFileStoreNameChooser(my_context_obj) 169 170 If you later want to change the way file ids are created from a 171 certain context, you only have to change the adapter implementation 172 accordingly. 173 174 Note, that this is only a convenience component. You don't have to 175 define context adapters but it makes things easier for others if you 176 do, as you don't have to remember the exact file id creation method 177 all the time and can change things quick and in only one location if 178 you need to do so. 179 180 Please see the :class:`FileStoreNameChooser` default implementation 181 below for details. 182 23 183 """ 24 184 import grok … … 180 340 class ImageStorage(grok.Container): 181 341 """A container for image files. 342 343 .. deprecated:: 0.2 344 345 Use :class:`waeup.sirp.ExtFileStore` instead. 182 346 """ 183 347 def _del(self): … … 246 410 247 411 248 class ExtFileStore(object): #grok.GlobalUtility):412 class ExtFileStore(object): 249 413 """External file store. 250 414 … … 408 572 def createFile(self, store, root, filename, file_id, f): 409 573 path = self.pathFromFileID(store, root, file_id) 410 return f, path, WAeUPImageFile(filename, file_id) 411 return path, HurryFile(filename, file_id) 574 return f, path, HurryFile(filename, file_id)
Note: See TracChangeset for help on using the changeset viewer.