« CITI blog | Main | git engineering for pNFS »

August 13, 2009

Roadmap for pNFS in the Linux kernel

In early 2008, we sketched out a road map for pNFS that tried to predict progress on NFSv4.1 implementation, standardization, and inclusion in the Linux kernel. Briefly, we predicted:

• Complete interoperable and functional implementations
• Convergence IETF Internet drafts

• NFSv4.1 RFC issued
• NFSv4.1 merged into mainline Linux kernel

• Developers tune pNFS performance at scale

This note looks at progress in adding NFSv4.1 to the Linux kernel. We're not far off track, maybe a couple months.

Linux kernels are not released on a specific schedule, but there is a discernible pattern.

When a kernel is released, a development kernel "opens up." Kernel maintainers then have a window of about two weeks to merge in major changes ready to see the light of day. The development kernel is then worked over for a couple months by maintainers. When the development kernel has stabilized, it is released, and the process starts anew.

The last several kernels were released on the following dates:

2.6.26 on July 13, 2008
2.6.27 on October 9. 2008
2.6.28 on December 24, 2008
2.6.29 on March 23, 2009
2.6.30 on June 9, 2009

This is consistent with a two and a half month cycle, with an extra two weeks over the winter holidays. So our best guess for the schedule of future releases is

2.6.31 in late August 2009
2.6.32 in early November 2009
2.6.33 in early February 2010

Pieces of NFSv4.1 are already present in the Linux kernel, in particular the sessions communication layer, mandatory in NFSv4.1, has a toehold: 2.6.30 has some preliminary server-side sessions code, although it lacks a few things:

• trunking
• back channel
• SSV (and some other security-related features)
• reboot recovery (no RECLAIM_COMPLETE)
• some miscellaneous — but mandatory — state operations, like DESTROY_CLIENTID and TEST_STATEID

None of the optional NFSv4.1 features, e.g., directory delegations, pNFS, and file delegation enhancements, are in 2.6.30.

Although the 2.6.31 is not yet released — it is "in stabilization" — the important stuff has already been accepted, so we know that 2.6.31 will have preliminary client-side sessions code, with more or less the same caveats as the 2.6.30 server-side sessions code.

When 2.6.32 opens up, we're expecting the sessions back channel — necessary for delegation and layout recalls — to be merged in. Developers have been testing this code at interoperability events, and it passes artificial tests, but it will need some TLC as 2.6.32 stabilizes before it can be used in production.

I'll write about the prospects for pNFS (i.e., layout ops) in my next post.

Posted by honey at August 13, 2009 01:14 PM



I'm glad to see things continue to progress!

The timeline was discussed at the pNFS BOF at SC08 last November; the slides are at www.pnfs.com. That timeline predicted "Linux distributions shipping supported pNFS in 2010". Do you still see that happening?

I ask because pNFS in a distro is probably a prerequisite to developers tuning pNFS "at scale", which to me means on the order of 1000 clients. IBM, for one, doesn't routinely keep idle development clusters of this size around, so such tuning would probably be at a customer site. Customers tend to run supported distros on their large clusters, often enterprise distros.

Posted by: roger@almaden.ibm.com at August 14, 2009 04:33 PM

Roger, I agree that support by the distros is critical. If fortune smiles on us all, then the mainline kernel will have complete pNFS support early in 2010. (I'm working on a blog post detailing that.) Whether that gives the distros enough time to do the necessary engineering is an important question.

As for enterprise distros, it seems to me that they lag somewhat (appropriately, imho), so ... who knows? At the end (or is it the start?) of the day, a lot will depend on customer demand.

Meanwhile, folks in PDSI are beginning to use large clusters (made available by partners in the national labs) to work on pNFS performance at scale. That work is just getting under way, but maybe we will have some early results to discuss at SC 09.

Posted by: honey at August 14, 2009 06:23 PM

Login to leave a comment. Create a new account.