W
Windows Apps Team
This post was written by Kraig Brockschmidt, Senior Program Manager in the Operating Systems Group
Lately we’ve heard some recurring questions from you about files and app data, so we’ve compiled a list of those questions and provided some brief answers here. Part 1 addressed questions about app data; in this second part we’ll cover your questions about files and the file system. Both posts apply equally to Windows 8.1 and Windows Phone 8.1.
Questions in this post:
Q. What file system locations can an app access programmatically without user consent?
A. By default, the only locations that an app can access without user consent are its package folder (read -only) and its app data folders (read-write). The package folder is accessed by using Windows.ApplicationModel.Package.Current.InstalledLocation, which is a StorageFolder object. The app data folders are accessed via the Windows.Storage.ApplicationData properties LocalFolder, RoamingFolder, and TemporaryFolder (and LocalCacheFolder on Windows Phone 8.1).
Note that when you have files in your Microsoft Visual Studio project that you want to include in your deployed app package, be sure to set their Copy to Output Dir properties in your project to Copy Always and set the Build Action to Content.
Q. How does the user give consent for other locations?
A. For media libraries—Pictures, Videos, Music—an app declares that it requests access through the associated capabilities in its manifest. These declarations appear on the app’s description page in the Windows Store and/or the Windows Phone Store, so that the user gives his or her consent through the act of installing the app. An app can also request programmatic access to removable storage through that capability; in that case, the app must also declare one or more specific file types, and its access is restricted to those file types. For all other locations, the user must explicitly grant access through the file or folder picker UI, or by launching a file or files through Windows Explorer. There are no exceptions to this rule.
Note that on Windows Phone 8.1, files obtained with the FileOpenPicker are read only; you must use the FileSavePicker to obtain write access to a file.
Q. What do StorageFile and StorageFolder objects represent, exactly?
A. StorageFile and StorageFolder objects (and their common base class, StorageItem), are best thought of as abstractions for traditional path names. That is, they are merely references to storage locations. They exist because file-like and folder-like entities can exist on locations other than the local file system—on OneDrive, on a third-party cloud storage provider, within another app, or other such places. Traditional path names are insufficient to refer to such other locations. Furthermore, the StorageFile and StorageFolder objects provide direct access to file and folder metadata, including cached thumbnails for all file types. Access to this metadata often eliminates the need to open such files to generate UI such as gallery experiences. Furthermore, StorageFile and StorageFolder objects maintain programmatic permissions (that is, user consent) for the entities they reference, as obtained through the file and folder pickers or through file launching.
Q. How does an app preserve programmatic access to files and folders across sessions?
A. To maintain the permissions maintained within a StorageFile or StorageFolder object, save the object by using the Windows.Storage.AccessCache class, either in the StorageApplicationPermission.MostRecentlyUsedList (a small set, limited to 25 for obvious reasons) or the general-purpose StorageApplicationPermission.FutureAccessList (limited to 1,000 objects). When you want to reopen the file or folder later, retrieve it from the AccessCache, which will restore permissions. Attempting to use textual path names for this purpose does not preserve permissions and will result in “access denied” exceptions. Note that if you want to preserve access to a large number of files in the same folder, it’s best to have the user select the StorageFolder as a whole, which you can then save as a single item in the FutureAccessList.
Q. When do I really need to open an image file vs. using thumbnails?
A. Most image files that belong to user have a much higher resolution than most device displays. So it’s almost never necessary to load the entire contents of an image file into memory just to shrink it down to fit into some part of an app’s UI, even for a full-screen view. Unless you’re loading the image file for pixel editing, then, use the StorageFile.GetThumbnailAsync or StorageFile.GetScaledImageAsThumbnailAsync methods to obtain a scaled rendering of the image file. This rendering—even for full-screen views—can often be done from pre-cached thumbnails, resulting in better performance. Furthermore, requesting a thumbnail at the exact size you require ensures that you consume only as much memory as necessary for an image of that size. Again, you get better overall performance and put less strain on low-memory systems.
Q. How do I make a StorageFile read-only (or set other file attributes)?
A. You change attributes via the StorageFile.properties property, which is a StorageItemContentProperties object. Create and populate a Windows.Foundation.Collections.PropertySet object with the properties you need, and pass it to StorageItemContentProperties.SavePropertiesAsync. To make a file read-only, specifically set the System.FileAttributes property with the value FILE_ATTRIBUTE_READONLY (or just 1) as described for the Win32 SetFileAttributes function.
Here’s a bit of code that checks the attribute on a file called test.jpg in the Pictures library and changes it to read only.
JavaScript
C#
Q. What’s the best way to check for the existence of a file?
A. On Windows, use the StorageFile.IsAvailable property or the StorageFolder.TryGetItemAsync method, which were both introduced in Windows 8.1 for this purpose. On Windows Phone 8.1, however, these members are not available, so currently you have to call StorageFolder.GetFileAsync within a try/catch block and check for “file not found” exceptions. (This approach also works on Windows.) Note that if you’re checking to determine whether to create the file, then just use StorageFolder.CreateFileAsync with CreateCollisionOption.OpenIfExists.
Q. What’s the best way to find specific file types in a folder?
A. Although you can call StorageFolder.GetFilesAsync and iterate through the collection to check for specific types, a better and more efficient way is to let the file system do the work by using a query created via StorageFolder.CreateFileQuery, CreateFolderQuery, and other variants. For finding specific file types, the best choice is CreateFileQueryWithOptions, which accepts a QueryOptionsobject that you can initialize with an array of file types.
Here’s some code to demonstrate that. The processResults function iterates the collection of StorageFile objects as needed, typically using thumbnails to display them.
JavaScript
C#
Note also that this approach has the added advantage of the StorageFileQueryResult.ContentsChangedevent, which will notify you if the contents of the queried folder have been changed. You then process the query results again without having to re-execute the query itself.
Q. How does an app access the user’s Documents library?
A. There are two parts to this answer. First, developers typically think about using the Documents library for storing arbitrary app data, but you should always store that in the app data folders instead. For arbitrary user data, it’s best to invoke the file picker or folder picker UI and let the user choose the save location (and again, you can retain permissions to the saved files through the AccessCache API). When invoking the pickers, you can specify the Documents library as the initial location, which is redirected to the user’s OneDrive folder by default. Beyond this, obtaining programmatic access to the Windows.Storage.KnownFolders.DocumentsLibraryrequires the Documents Library declaration in the manifest, and that declaration is allowed only for apps that are submitted to the Store with written justification under a company (not an individual) account. Even then, access is restricted to separately declared file types. In short, there are only very limited scenarios where programmatic access is warranted, so most apps should invoke the picker instead.
Q. How do I close a StorageFile object when there’s no Close method?
A. Remember that a StorageFile object is an abstraction for a path name. This means that obtaining a StorageFile object, even with a method like StorageFolder.GetFileAsync, does not actually open the file and lock access—it merely holds a reference to that file. A file is not open until you acquire a stream for it, and to close that file you dispose of the stream (see the next question).
Q. How do streams relate to StorageFile objects?
A. If you think of a StorageFile object as an abstraction for a path name, a stream is an abstraction for a traditional file handle. Streams are how the Windows Runtime manages latency differences between different storage systems like memory, disk, and network. For files, calling StorageFile.OpenAsync (or OpenReadAsync, OpenSequentialReadAsync, and OpenTransactedWriteAsync) is what produces a stream, and it’s the existence of the stream that keeps the file open and possibly prevents access by other apps. To close the stream, call its Close method (followed by Dispose in C#, VB, and C++). You can also scope its use with a using statement (C# or VB), or simply wrap a DataReader or DataWriter class around it and call that object’s Close method.
Q. I’ve looked at file handling through streams and it all looks rather complicated. Is there an easier way?
A. Yes. Check out the Windows.Storage.FileIO and Windows.Storage.PathIO helper classes, which encapsulate the most common file operations into much simpler APIs. You can still use low-level streams for precise control when you need it, but for most scenarios the FileIO and PathIO helpers are sufficient. To give an idea of the difference, a routine to write a string to a file in the Temporary app data folder needs about 15 lines of code involving at least four asynchronous function calls, and that’s fine if you need to inject specific actions into any part of the process. But with the FileIO class, you can get the same result in about 4 lines of code, and with PathIO you can do it in a single line.
Q. What are buffers and what on earth do I do with them?
A. Reading from a stream produces an object called a Buffer, which is all well and good until you look at the documentation and see that the Buffer class has only two properties—Length and Capacity. So how do you get the data out of one? That’s what the Windows.Storage.DataReader class is for. You create an instance of it with the static method DataReader.FromBuffer, passing in the Buffer object, after which you can call the DataReader class’s methods like (ReadBytes and ReadString) as needed. Similarly, if you want to write data to a stream, you’ll need to create a buffer. To do that, create an instance of Windows.Storage.DataWriter, call its WriteBytes, WriteString, and other such methods, and then call DetachBuffer to obtain the Buffer object. And if you want to create a Buffer from an existing byte array, check out Windows.Security.Cryptography.CryptographicBuffer.CreateFromByteArray (which actually has nothing to do with cryptography per se).
That said, there’s also a secret back door—well, it’s documented, but not well known: the Buffer object also supports a COM interface called IBufferByteAccess, whose single IBufferByteAccess::Buffer method returns the whole byte array. You have to call IBuffer::QueryInterface directly to request this, which is something you have to do from C++. So if you’re writing a Windows Runtime component or app in C++ for high-performance buffer manipulation, it’s good to know about this method.
Resources:
Continue reading...
Lately we’ve heard some recurring questions from you about files and app data, so we’ve compiled a list of those questions and provided some brief answers here. Part 1 addressed questions about app data; in this second part we’ll cover your questions about files and the file system. Both posts apply equally to Windows 8.1 and Windows Phone 8.1.
Questions in this post:
- What file system locations can an app access programmatically without user consent?
- How does the user give consent for other locations?
- What do StorageFile and StorageFolderobjects represent, exactly?
- How does an app preserve programmatic access to files and folders across sessions?
- When do I really need to open an image file vs. using thumbnails?
- How do I make a StorageFileread-only (or set other file attributes)?
- What’s the best way to check for the existence of a file?
- What’s the best way to find specific file types in a folder?
- How does an app access the user’s Documents library?
- How do I close a StorageFile object when there’s no Closemethod?
- How do streams relate to StorageFileobjects?
- I’ve looked at file handling through streams and it all looks rather complicated. Is there an easier way?
- What are buffers and what on earth do I do with them?
Q. What file system locations can an app access programmatically without user consent?
A. By default, the only locations that an app can access without user consent are its package folder (read -only) and its app data folders (read-write). The package folder is accessed by using Windows.ApplicationModel.Package.Current.InstalledLocation, which is a StorageFolder object. The app data folders are accessed via the Windows.Storage.ApplicationData properties LocalFolder, RoamingFolder, and TemporaryFolder (and LocalCacheFolder on Windows Phone 8.1).
Note that when you have files in your Microsoft Visual Studio project that you want to include in your deployed app package, be sure to set their Copy to Output Dir properties in your project to Copy Always and set the Build Action to Content.
Q. How does the user give consent for other locations?
A. For media libraries—Pictures, Videos, Music—an app declares that it requests access through the associated capabilities in its manifest. These declarations appear on the app’s description page in the Windows Store and/or the Windows Phone Store, so that the user gives his or her consent through the act of installing the app. An app can also request programmatic access to removable storage through that capability; in that case, the app must also declare one or more specific file types, and its access is restricted to those file types. For all other locations, the user must explicitly grant access through the file or folder picker UI, or by launching a file or files through Windows Explorer. There are no exceptions to this rule.
Note that on Windows Phone 8.1, files obtained with the FileOpenPicker are read only; you must use the FileSavePicker to obtain write access to a file.
Q. What do StorageFile and StorageFolder objects represent, exactly?
A. StorageFile and StorageFolder objects (and their common base class, StorageItem), are best thought of as abstractions for traditional path names. That is, they are merely references to storage locations. They exist because file-like and folder-like entities can exist on locations other than the local file system—on OneDrive, on a third-party cloud storage provider, within another app, or other such places. Traditional path names are insufficient to refer to such other locations. Furthermore, the StorageFile and StorageFolder objects provide direct access to file and folder metadata, including cached thumbnails for all file types. Access to this metadata often eliminates the need to open such files to generate UI such as gallery experiences. Furthermore, StorageFile and StorageFolder objects maintain programmatic permissions (that is, user consent) for the entities they reference, as obtained through the file and folder pickers or through file launching.
Q. How does an app preserve programmatic access to files and folders across sessions?
A. To maintain the permissions maintained within a StorageFile or StorageFolder object, save the object by using the Windows.Storage.AccessCache class, either in the StorageApplicationPermission.MostRecentlyUsedList (a small set, limited to 25 for obvious reasons) or the general-purpose StorageApplicationPermission.FutureAccessList (limited to 1,000 objects). When you want to reopen the file or folder later, retrieve it from the AccessCache, which will restore permissions. Attempting to use textual path names for this purpose does not preserve permissions and will result in “access denied” exceptions. Note that if you want to preserve access to a large number of files in the same folder, it’s best to have the user select the StorageFolder as a whole, which you can then save as a single item in the FutureAccessList.
Q. When do I really need to open an image file vs. using thumbnails?
A. Most image files that belong to user have a much higher resolution than most device displays. So it’s almost never necessary to load the entire contents of an image file into memory just to shrink it down to fit into some part of an app’s UI, even for a full-screen view. Unless you’re loading the image file for pixel editing, then, use the StorageFile.GetThumbnailAsync or StorageFile.GetScaledImageAsThumbnailAsync methods to obtain a scaled rendering of the image file. This rendering—even for full-screen views—can often be done from pre-cached thumbnails, resulting in better performance. Furthermore, requesting a thumbnail at the exact size you require ensures that you consume only as much memory as necessary for an image of that size. Again, you get better overall performance and put less strain on low-memory systems.
Q. How do I make a StorageFile read-only (or set other file attributes)?
A. You change attributes via the StorageFile.properties property, which is a StorageItemContentProperties object. Create and populate a Windows.Foundation.Collections.PropertySet object with the properties you need, and pass it to StorageItemContentProperties.SavePropertiesAsync. To make a file read-only, specifically set the System.FileAttributes property with the value FILE_ATTRIBUTE_READONLY (or just 1) as described for the Win32 SetFileAttributes function.
Here’s a bit of code that checks the attribute on a file called test.jpg in the Pictures library and changes it to read only.
JavaScript
var key = "System.FileAttributes";
var FILE_ATTRIBUTES_READONLY = 1;
var file;
var pix = Windows.Storage.KnownFolders.picturesLibrary;
pix.getFileAsync("test.jpg").then(function (fileTemp) {
file = fileTemp;
return file.properties.retrievePropertiesAsync([key]);
}).done(function (props) {
if (props) {
props[key] |= FILE_ATTRIBUTES_READONLY;
} else {
props = new Windows.Foundation.Collections.PropertySet();
props.insert(key, FILE_ATTRIBUTES_READONLY);
}
file.properties.savePropertiesAsync(props);
});
var FILE_ATTRIBUTES_READONLY = 1;
var file;
var pix = Windows.Storage.KnownFolders.picturesLibrary;
pix.getFileAsync("test.jpg").then(function (fileTemp) {
file = fileTemp;
return file.properties.retrievePropertiesAsync([key]);
}).done(function (props) {
if (props) {
props[key] |= FILE_ATTRIBUTES_READONLY;
} else {
props = new Windows.Foundation.Collections.PropertySet();
props.insert(key, FILE_ATTRIBUTES_READONLY);
}
file.properties.savePropertiesAsync(props);
});
C#
String key = "System.FileAttributes";
UInt32 FILE_ATTRIBUTES_READONLY = 1;
var pix = Windows.Storage.KnownFolders.PicturesLibrary;
var file = await pix.GetFileAsync("test.jpg");
String[] retrieveList = new String[] { key };
var props = await file.Properties.RetrievePropertiesAsync(retrieveList);
if (props != null)
{
var temp = (UInt32)props[key] | FILE_ATTRIBUTES_READONLY;
props[key] = temp;
}
else
{
props = new Windows.Foundation.Collections.PropertySet();
props.Add(key, FILE_ATTRIBUTES_READONLY);
}
await file.Properties.SavePropertiesAsync(props);
UInt32 FILE_ATTRIBUTES_READONLY = 1;
var pix = Windows.Storage.KnownFolders.PicturesLibrary;
var file = await pix.GetFileAsync("test.jpg");
String[] retrieveList = new String[] { key };
var props = await file.Properties.RetrievePropertiesAsync(retrieveList);
if (props != null)
{
var temp = (UInt32)props[key] | FILE_ATTRIBUTES_READONLY;
props[key] = temp;
}
else
{
props = new Windows.Foundation.Collections.PropertySet();
props.Add(key, FILE_ATTRIBUTES_READONLY);
}
await file.Properties.SavePropertiesAsync(props);
Q. What’s the best way to check for the existence of a file?
A. On Windows, use the StorageFile.IsAvailable property or the StorageFolder.TryGetItemAsync method, which were both introduced in Windows 8.1 for this purpose. On Windows Phone 8.1, however, these members are not available, so currently you have to call StorageFolder.GetFileAsync within a try/catch block and check for “file not found” exceptions. (This approach also works on Windows.) Note that if you’re checking to determine whether to create the file, then just use StorageFolder.CreateFileAsync with CreateCollisionOption.OpenIfExists.
Q. What’s the best way to find specific file types in a folder?
A. Although you can call StorageFolder.GetFilesAsync and iterate through the collection to check for specific types, a better and more efficient way is to let the file system do the work by using a query created via StorageFolder.CreateFileQuery, CreateFolderQuery, and other variants. For finding specific file types, the best choice is CreateFileQueryWithOptions, which accepts a QueryOptionsobject that you can initialize with an array of file types.
Here’s some code to demonstrate that. The processResults function iterates the collection of StorageFile objects as needed, typically using thumbnails to display them.
JavaScript
var picturesLibrary = Windows.Storage.KnownFolders.picturesLibrary;
var options = new Windows.Storage.Search.QueryOptions(
Windows.Storage.Search.CommonFileQuery.orderByName, [".jpg", ".jpeg", ".png", ".gif"]);
// Could also use orderByDate instead of orderByName.
if (picturesLibrary.areQueryOptionsSupported(options)) {
var query = picturesLibrary.createFileQueryWithOptions(options);
query.getFilesAsync().done(function (files) {
processResults(files);
});
}
var options = new Windows.Storage.Search.QueryOptions(
Windows.Storage.Search.CommonFileQuery.orderByName, [".jpg", ".jpeg", ".png", ".gif"]);
// Could also use orderByDate instead of orderByName.
if (picturesLibrary.areQueryOptionsSupported(options)) {
var query = picturesLibrary.createFileQueryWithOptions(options);
query.getFilesAsync().done(function (files) {
processResults(files);
});
}
C#
var picturesLibrary = Windows.Storage.KnownFolders.PicturesLibrary;
String[] types = {".jpg", ".jpeg", ".png", ".gif"};
var options = new Windows.Storage.Search.QueryOptions(Windows.Storage.Search.CommonFileQuery.OrderByName, types);
// Could also use OrderByDate instead of OrderByName.
if (picturesLibrary.AreQueryOptionsSupported(options))
{
var query = picturesLibrary.CreateFileQueryWithOptions(options);
processResults(await query.GetFilesAsync());
}
String[] types = {".jpg", ".jpeg", ".png", ".gif"};
var options = new Windows.Storage.Search.QueryOptions(Windows.Storage.Search.CommonFileQuery.OrderByName, types);
// Could also use OrderByDate instead of OrderByName.
if (picturesLibrary.AreQueryOptionsSupported(options))
{
var query = picturesLibrary.CreateFileQueryWithOptions(options);
processResults(await query.GetFilesAsync());
}
Note also that this approach has the added advantage of the StorageFileQueryResult.ContentsChangedevent, which will notify you if the contents of the queried folder have been changed. You then process the query results again without having to re-execute the query itself.
Q. How does an app access the user’s Documents library?
A. There are two parts to this answer. First, developers typically think about using the Documents library for storing arbitrary app data, but you should always store that in the app data folders instead. For arbitrary user data, it’s best to invoke the file picker or folder picker UI and let the user choose the save location (and again, you can retain permissions to the saved files through the AccessCache API). When invoking the pickers, you can specify the Documents library as the initial location, which is redirected to the user’s OneDrive folder by default. Beyond this, obtaining programmatic access to the Windows.Storage.KnownFolders.DocumentsLibraryrequires the Documents Library declaration in the manifest, and that declaration is allowed only for apps that are submitted to the Store with written justification under a company (not an individual) account. Even then, access is restricted to separately declared file types. In short, there are only very limited scenarios where programmatic access is warranted, so most apps should invoke the picker instead.
Q. How do I close a StorageFile object when there’s no Close method?
A. Remember that a StorageFile object is an abstraction for a path name. This means that obtaining a StorageFile object, even with a method like StorageFolder.GetFileAsync, does not actually open the file and lock access—it merely holds a reference to that file. A file is not open until you acquire a stream for it, and to close that file you dispose of the stream (see the next question).
Q. How do streams relate to StorageFile objects?
A. If you think of a StorageFile object as an abstraction for a path name, a stream is an abstraction for a traditional file handle. Streams are how the Windows Runtime manages latency differences between different storage systems like memory, disk, and network. For files, calling StorageFile.OpenAsync (or OpenReadAsync, OpenSequentialReadAsync, and OpenTransactedWriteAsync) is what produces a stream, and it’s the existence of the stream that keeps the file open and possibly prevents access by other apps. To close the stream, call its Close method (followed by Dispose in C#, VB, and C++). You can also scope its use with a using statement (C# or VB), or simply wrap a DataReader or DataWriter class around it and call that object’s Close method.
Q. I’ve looked at file handling through streams and it all looks rather complicated. Is there an easier way?
A. Yes. Check out the Windows.Storage.FileIO and Windows.Storage.PathIO helper classes, which encapsulate the most common file operations into much simpler APIs. You can still use low-level streams for precise control when you need it, but for most scenarios the FileIO and PathIO helpers are sufficient. To give an idea of the difference, a routine to write a string to a file in the Temporary app data folder needs about 15 lines of code involving at least four asynchronous function calls, and that’s fine if you need to inject specific actions into any part of the process. But with the FileIO class, you can get the same result in about 4 lines of code, and with PathIO you can do it in a single line.
Q. What are buffers and what on earth do I do with them?
A. Reading from a stream produces an object called a Buffer, which is all well and good until you look at the documentation and see that the Buffer class has only two properties—Length and Capacity. So how do you get the data out of one? That’s what the Windows.Storage.DataReader class is for. You create an instance of it with the static method DataReader.FromBuffer, passing in the Buffer object, after which you can call the DataReader class’s methods like (ReadBytes and ReadString) as needed. Similarly, if you want to write data to a stream, you’ll need to create a buffer. To do that, create an instance of Windows.Storage.DataWriter, call its WriteBytes, WriteString, and other such methods, and then call DetachBuffer to obtain the Buffer object. And if you want to create a Buffer from an existing byte array, check out Windows.Security.Cryptography.CryptographicBuffer.CreateFromByteArray (which actually has nothing to do with cryptography per se).
That said, there’s also a secret back door—well, it’s documented, but not well known: the Buffer object also supports a COM interface called IBufferByteAccess, whose single IBufferByteAccess::Buffer method returns the whole byte array. You have to call IBuffer::QueryInterface directly to request this, which is something you have to do from C++. So if you’re writing a Windows Runtime component or app in C++ for high-performance buffer manipulation, it’s good to know about this method.
Resources:
- User Data Management in Windows Phone, by Tony Pendolino and Wes Peter, from //build 2014.
- What’s New for Working with Files by Marc Wautier, from //build 2014.
- Chapter 11 of the free ebook from Microsoft Press, Programming Windows Store Apps with HTML, CSS, and JavaScript, 2nd Edition, by Kraig Brockschmidt, which despite its title is a very helpful resource for developers in all programming languages.
Continue reading...