OLD | NEW |
(Empty) | |
| 1 Handling of file:// URIs: |
| 2 |
| 3 This directory contains code to map basic boto connection, bucket, and key |
| 4 operations onto files in the local filesystem, in support of file:// |
| 5 URI operations. |
| 6 |
| 7 Bucket storage operations cannot be mapped completely onto a file system |
| 8 because of the different naming semantics in these types of systems: the |
| 9 former have a flat name space of objects within each named bucket; the |
| 10 latter have a hierarchical name space of files, and nothing corresponding to |
| 11 the notion of a bucket. The mapping we selected was guided by the desire |
| 12 to achieve meaningful semantics for a useful subset of operations that can |
| 13 be implemented polymorphically across both types of systems. We considered |
| 14 several possibilities for mapping path names to bucket + object name: |
| 15 |
| 16 1) bucket = the file system root or local directory (for absolute vs |
| 17 relative file:// URIs, respectively) and object = remainder of path. |
| 18 We discarded this choice because the get_all_keys() method doesn't make |
| 19 sense under this approach: Enumerating all files under the root or current |
| 20 directory could include more than the caller intended. For example, |
| 21 StorageUri("file:///usr/bin/X11/vim").get_all_keys() would enumerate all |
| 22 files in the file system. |
| 23 |
| 24 2) bucket is treated mostly as an anonymous placeholder, with the object |
| 25 name holding the URI path (minus the "file://" part). Two sub-options, |
| 26 for object enumeration (the get_all_keys() call): |
| 27 a) disallow get_all_keys(). This isn't great, as then the caller must |
| 28 know the URI type before deciding whether to make this call. |
| 29 b) return the single key for which this "bucket" was defined. |
| 30 Note that this option means the app cannot use this API for listing |
| 31 contents of the file system. While that makes the API less generally |
| 32 useful, it avoids the potentially dangerous/unintended consequences |
| 33 noted in option (1) above. |
| 34 |
| 35 We selected 2b, resulting in a class hierarchy where StorageUri is an abstract |
| 36 class, with FileStorageUri and BucketStorageUri subclasses. |
| 37 |
| 38 Some additional notes: |
| 39 |
| 40 BucketStorageUri and FileStorageUri each implement these methods: |
| 41 - clone_replace_name() creates a same-type URI with a |
| 42 different object name - which is useful for various enumeration cases |
| 43 (e.g., implementing wildcarding in a command line utility). |
| 44 - names_container() determines if the given URI names a container for |
| 45 multiple objects/files - i.e., a bucket or directory. |
| 46 - names_singleton() determines if the given URI names an individual object |
| 47 or file. |
| 48 - is_file_uri() and is_cloud_uri() determine if the given URI is a |
| 49 FileStorageUri or BucketStorageUri, respectively |
OLD | NEW |