Folder Redirection – Denial of Service Waiting to Happen
This article is part of Helge’s Profile Toolkit, a set of posts explaining the knowledge and tools required to tame Windows user profiles.
One of the problems inherent to roaming user profiles is that they are copied to the local computer during logon. That takes time. Hence the motivation to limit the size of user profiles. One way of doing that is to put a quota on them, another one is folder redirection. Although widely used, redirection has its problems; problems so grave that Shawn Bass started to campaign against using folder redirection altogether. In this article I describe why Shawn despises redirection, which alternatives are available now what we can hope for in the future.
The Problem
User profiles tend to grow over time. Growth is caused by users putting files in their Documents or similar folder and by applications storing settings in the roaming AppData directory. Growth by itself is natural and totally OK. It gets problematic, though, when we introduce another variable: stateless systems.
Terminal servers and many virtual desktops are stateless. They store user data only temporarily and are reset to a pristine state as soon as the last user session ends. As a consequence, the device cannot be used to store data permanently. Everything must reside on file servers and either be accessed over the network or be copied temporarily to the local hard disk. Home directories are an example for the former, user profiles an example for the latter.
Copying user profiles to the local hard disk slows logons, potentially dramatically (since the logon process is halted until the copy is finished). Slow logons on a PC that is booted up in the morning and shut down in the evening are not so bad, but published terminal server applications are used much more dynamically than that. Session setup happens every time a user starts an application, and the user profile is loaded into each session. Delays are much more critical here. Thus the motivation to keep user profiles small.
Folder Redirection
One way to do that is folder redirection. It allows administrators to move certain parts of the profile to a different location, typically the home directory. Since less data is stored in the profile, it stays small and logons are fast. By redirecting from the local hard disk to the network we gain logon speed, but we lose – reliability. Files are not fetched from the local hard disk any more (a simple and pretty reliable operation). They travel over the network instead, a much more complex process. For it to happen smoothly, both the network and the file server need to be in tip-top shape and not too heavily loaded. Any “hiccups” may cause frozen desktops and applications. As Shawn Bass put it, IO delays may cause application denial of service.
To understand the gravity of the problem, one needs to know why applications do not handle network delays gracefully. It really is simple: because applications do not know that they are fetching data from the network. Folder redirection is transparent to applications. When a developer writes code that accesses a file in the user profile, he assumes the file is stored on the local hard disk and does not plan for significant delays like he would if he were accessing a file on the network. As a result, the GUI thread is blocked until the IO has completed. And a blocked GUI thread means a frozen application.
Many people think that PCs are pretty much independent of the backend infrastructure – with folder redirection they are not! When there are problems in the backend, the users cannot work any more.
The Solution (as it Exists Today)
So what do we do? If I have not scared you enough, if you have great networks and file servers or if you just do not care, you continue to use folder redirection. In all other cases, you should at least consider alternatives. Unfortunately I am only aware of one, and it is not perfect yet either: profile streaming.
If you do not wait for the copying, session setup is fast and folder redirection is not necessary. Profile streaming does just that: it moves the copying from the foreground to the background. Asynchronous instead of synchronous. The logon process can continue while the user profile is being copied from the file server. That sounds too good to be true, and it is (partially). You should be aware of the following downsides:
- Profile streaming is available from few vendors only. I am aware of AppSense, Citrix, RES and VMware.
- It adds complexity. There is more that can go wrong.
- It does not reduce the load on the file server during peak logon times.
- Profile size still matters, since the entire profile needs to be copied to the machine. Very big profiles may overload the file server during peak logon times (see previous point).
The Solution (as it Should Be)
Profile streaming needs to evolve and solve the last two remaining technical problems, too: profile size and file server load during logon. The way to do that is to identify when files were used last. If that information is available for each file, then a policy can be implemented to only copy the files that have been used in the last six months, for example. Everything else would be streamed on demand. This way dependency on the network is limited to files that are unlikely to be accessed and responsiveness of the applications that are used regularly is guaranteed.
Implementing what might be called profile streaming 2.0 should not be too hard. I really hope to see this in a product some day. As an added benefit, it would remove the need to use profile quotas (something I am not too happy with).
Conclusion
User profiles are difficult. There is no perfect solution. Be aware of potential problems and use what works best for you.
7 Comments
Fantastic article Helge, I only wish there was a native solution for the problem without relying on third parties.
For reference RES workspace manager can also stream profile components.
A
Thanks! I added RES to the list.
Hi,
Firstly a disclaimer – I work for AppSense.
A very interesting article, however I would like to clarify a couple of points about how AppSense Environment Manager does “profile streaming”.
You state about profile streaming in general:
• It does not reduce the load on the file server during peak logon times.
• Profile size still matters, since the entire profile needs to be copied to the machine. Very big profiles may overload the file server during peak logon times (see previous point).
With AppSense Environment Manager only a certain amount of data is read form the Personalization server during logon, things like desktop backgrounds, language settings, keyboard layout etc… – we call these things “Desktop Settings”, “Session Data” & “Certificates”.
Application data is only copied from the Personalization server when an application is started, when the application closes the data is copied back to the Personalization server – assuming something has changed.
If you imagine a scenario where you have 20 applications all storing some data in the user profile, when using a roaming profile all the data will be copied in at logon and out a logoff. However when using AppSense Environment Manager my application settings are only copied in when I run the application, if I don’t run the application during that session the data is not copied in.
So to answer the first point – load can be reduced during peak logon times by using AppSense Environment Manager, there will still be some load, but only what you need to get the session up and running.
For the second point – I agree profile size does matter and servers can get overloaded during logon times however when using AppSense Environment Manager the entire profile is not copied at logon so you can reduce this load.
Cheers,
Rob
Another great article.
I think the only way to get around this issue is to address it at the application level – applications like Google Chrome, Evernote and, dare I say it, even Microsoft OneNote, are using “cloud” syncing to fix the data issues. Unfortunately, the only app in that list that has worked out syncing preferences is Chrome.
Microsoft has a legacy and its support for that legacy is both one of its biggest advantages and one of its biggest problems. Profiles are a great example of this paradox. In order to support the enterprise, it must be a feature that old apps still work and old apps depend on the profile. To say, as Aaron does above, that the solution is in the hands of the application vendors is frankly ridiculous. The solution has to be in the platform or a layer on top of the platform that applies to all apps.
Roaming profiles and the introduction of folder redirection are an attempt by Microsoft to solve the issue that we no longer just sit at the same device. The splitting of AppData into Local, Locallow and Roaming is yet another fix onto an essentially broken system. Mostly because either the app isn’t being developed any more or the devs ignore it completely.
As to Shawns point, if you assume that a user needs more than a local profile (not always the case) you need a backend layer of some kind, in which case you have to rely on that infrastructure, the question then becomes how that backend infrastructure is architected.
So what is the solution? Well as I’ve heard you say before Helge, streaming is the only solution. it may be the only solution now, but it is still not good enough. The solution needs to learn the most accessed files and not just stream on demand, but background download the rest, with QoS of both the network, the underlying storage and the providing storage in mind.
Lets face it though that is just for Microsoft profiles, I think it gets even worse when you put those profiles into ‘the cloud’. There is not a single unified way of traversing machines with cloud personalisation, dropbox doesn’t care about Chrome, which doesn’t care about OneNote etc. What relying on cloud services does is to fragment the responsibility for settings till it is unmanageable and it is just going to get worse.
Until we have a platform that makes sense in the modern world to build on, that is open for everybody to use, and we can address control and support in a meaningful way these conversations are going to arise constantly. Whether Microsoft or Google or a third party like Appsense or RES etc provide it is immaterial. Until then this is going to be a thorn in our sides and we better get used to applying band aids.
This is an old thread but felt I would comment anyway. Great that we are looking for solutions to this. However we are enterprise environments and we need to not only consider the technical requirements here but also security and privacy considerations. I struggle that we should allow any local data to exist on a workstation. I remember a time when Zenworks applied Volatile User Profiles. Profile created at login and removed at logout.
This is something that I do not see Microsoft supporting at the moment. At least not cleanly
I also assume most of us are not backing up the local workstations. Again an argument that nothing should be stored there.
I am not writing to criticize any of the existing statements. Only to vent that security is such a hard topic to solve. That without some serious infrastructure like VDI and RDS its difficult to protect our end users from themselves.