SNIA Experts on Data

The World of Storage Management: Insights and Standards

• SNIA • Episode 2

Share episode

Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.

0:00 | 26:55

Philip Alsop from Digitalisation World interviews Richelle Ahlvers, Chair of the SNIA Storage Management Initiative. They discuss what’s new with SNIA Swordfish®, the storage management standard, the latest developments on work with industry partner organizations, the new SNIA Swordfish Conformance Testing Program and Next-Gen Lab.

For more information on SNIA Swordfish, check out the SNIA Educational Library. 

SNIA is an industry organization that develops global standards and delivers vendor-neutral education on technologies related to data.  In these interviews, SNIA experts on data cover a wide range of topics on both established and emerging technologies.

About SNIA:

Kristin Hauser

Welcome to the SNEA on Data Podcast. Each episode highlights key technologies related to handling and optimizing data. This is SNEA on Data Podcast, a storage management update from SNEA, where Rochelle Alvers, chair of the SNEA Storage Management Initiative, discusses the latest SNEA storage management developments and highlights what's new in the SNEA Stored Fish standard.

Philip Alsop

Today we're going to have a little update on the SNEA Storage Network Industry Association Storage Management Initiative. I think we're going to talk mainly about the swordfish management standard, storage management standard. Perhaps, for those that aren't as familiar with it as others, can you just give us a very brief resume of what that standard is, what it does?

Richelle Ahlvers

Sure, the SNEA Swordfish Spec is actually an extension of DMTF's Redfish Specifications. Our spec itself really just focuses on adding all of the storage specific extensions. We let DMTF worry about protocols and all of the underlying bits and bobs. That lets us just focus on all of the actual storage specific pieces. You focus on all of the configurations for anything basically above a drive that's directly attached to a single drive a single or one or two drives directly attached to a server. Anything above that is covered by swordfish. We work with everyone from direct attached configurations that are building HBAs and raid cards and have anything more than a very simple configuration.

Richelle Ahlvers

Anytime they start to get very big, you want to go to swordfish. We're also doing a lot of work with, obviously, external storage things like that file systems that we've done. Historically, A lot of our recent work in the last couple of years has also been focusing on making sure we have excellent support and are basically the management interface for NVM Express and NVM over fabrics. While they have direct interfaces when you want to bring them up and manage them at scale in an enterprise data center attaching your systems. That's where you want to bring in swordfish. We partner with the Redfish quite extensively, so you get comprehensive management across server storage fabrics.

Philip Alsop

You mentioned NVM Express there and also the fact that you work with other. Can you just give us an outline what you do with them and presumably some other industry partner organizations that you do stuff with Absolutely?

Richelle Ahlvers

Let me start with our partnership with both DMTF and NVM Express. As I mentioned, we extend Redfish when we started working with NVM Express. It's actually a three-way agreement. We basically lead the work bringing NVM support into server management and storage management, because there are some cases where NVM are going to be very simple. What we've done over the last couple of years is basically taken and put the two specs side by side or actually on top of each other. Redfish and swordfish you bring in at the top layer with your enterprise and your data system and your system management. You have NVM, the base specs, coming up from the devices over your PCI bus at this level and we basically built the bridge between those two things.

Richelle Ahlvers

We've done everything from mapping all the schema properties between one and the other, as well as also writing documentation. One of our pieces of collateral is we call the mapping guide. That allows somebody who's not necessarily super familiar with one or the other to come in and be able to say, okay, this property where does that correspond to? Over here We've got lots of material. It talks about how to manage NVM devices.

Richelle Ahlvers

There are storage devices, but there's a few differences from the way we've done traditional storage management. We've mapped all of those things together. You have seamless management and standards based all the way up from your NVM devices using admin commands or MI out of band, and then putting Redfish and Swordfish on top of that. I will frequently say Redfish and Swordfish because to us it's all one thing. We cover the storage portions of the schema and Redfish covers a lot of the base schema. We extend wherever we need to. We try not to re-indicate wheels anywhere, but we focus on all those storage specific use cases like the NVM environment, which is why NVM Express came to us as a way to lead that whole mapping effort.

Philip Alsop

Okay, and I'm guessing a logical extension, which I think is fairly new to you, is the performance test program which, in simple terms, presumably is where you're checking that everything works together. And does you know if a particular manufacturer says it does this, it does actually conform to your standard, et cetera? Can you just give us some oh, that's probably yeah, wildly accurate, but in simple, but give us the actual. You know how does it work?

Richelle Ahlvers

It's a very good simplification. So SNEA has actually had a performance test programs for management software for many years. We've had the SMI Ask a Formance test program, and so we've basically taken that same concept and applied it to sort fish. So what we've done in order to be able to enable that is we've developed what we call a set of profiles, and a profile is basically a written agreement, effectively between the clients and servers that say this is the functionality that's going to be, that we're going to implement as at minimum for the standard, and that's what we test through the performance test program. And so we have a set of open source based tools. Redfish actually uses a lot of these same tools, a lot of the same infrastructure, but they don't have a formalized program around it like SNEA does.

Richelle Ahlvers

We found this to be much more important on the storage site historically, and so what we've got basically are a set of about. You know you could kind of think of them as little recipes that you can build and add on top of, but we've got about 20 different profiles that you, so we, that implementations can pick from and say I want to do this set of features, I want to do this set of features and then they can test those through these sort fish conformance test program and then advertise, you know, both, compliance, vendor, neutral compliance test validation hey, it wasn't just us. You know SNEA says we did this too as well as then the clients can see specifically what's in there. We also have a runtime feature that allows someone to tell at runtime what, what, what specifically of those features they're supporting. So they can tell both at runtime or they can go check the website and verify that that those tests actually get passed. So we're very proud of that.

Richelle Ahlvers

This we've developed, like I said, this whole thing on an open source based reference implementation. We have a couple of companies working with the program now and what they're doing is they're actually not waiting until the end of you know that till their, their products are complete and trying to start testing. They're actually using the program and the test framework while they're doing development. So as they're developing, they're saying is that, hey, are, is this what we're proposing to do going to pass and we can run through and check and give them feedback and so, as they iterate through their whole development cycle to get their products out, they're able to, you know, get closer and closer to you know and you know and help them accelerate their, their development, at the same time ensuring that they're building the right thing. So that's one of the features that that participating with our interoperability lab and our conformance test program early can offer.

Philip Alsop

I think you said there are about 20 proposals. Is that, for the time being, everything you visit? Or is that the first tranche and you know there's going to be some more you've got to bring? Or will they just happen as new technologies and vendors do different things? How do you see that?

Richelle Ahlvers

Yes, good question. This is our second wave. I do envision that there will be more and customers can also kind of create their own based on kind of a pick and choose. So the way these are structured is we have a set of base functionality profiles and these include everything from base discovery. Am I advertising myself correctly to block provisioning? What are the requirements that I have to do in order to say I can advertise that I support provisioning? And then we have, you know, once they get a little bit more complicated replication, all of these kinds of base features that you'd expect from storage devices.

Richelle Ahlvers

And then we have a whole set we've developed for NVMe as well. There's an NVMe drive, there's an NVMe Ethernet attached drive, and so, again, these are cumulative. Those Ethernet attached drive, doesn't? It just references the drive one as the base one and adds a few more requirements. So as implementers are going through, they can basically say well, I know I built an Ethernet attached drive, but I also support provisioning. So I'm just going to also say that I support that one, this other little profile, and bring that in, so we don't have to basically come up with, you know, these huge, long, completely independent ones. They're all these nice little chunks what we've been doing in the last, this last release, version 1.2.4, that we're actually just in the middle of releasing right now.

Richelle Ahlvers

We've added about eight new profiles in this bundle. They include getting into a little bit more detail, functionality for implementations, things like access rights management, connectivity management, as we especially as we get into NVMe over fabrics is actually the same fabric requirements as for external storage devices connecting to fabrics. So we've made sure that those are standardized. But we've also specified here's the requirement, the base requirements and the functionality for how you do that. We've also then gone back in and added some what we call some aggregation style profiles.

Richelle Ahlvers

So we have a couple things that are all they're. They don't really have a lot of functionality in and of themselves, but they're basically. You know, I want those four profiles and that's your requirements. And these are good for things like J-Bofs and E-Bofs, where all of the components are something else. But we can basically say, if you want it, you're going to build an NVMe, e-bof. Your actual requirements are include, you know, include these four profiles, and that's all nice and wrapped up into a single, single descriptor then. So those are the kinds of things we expect to keep building. I we're, we'd like to enter doing some work to reach out to the OCP organization on trying to extend. They've got server based profiles now and we're we're looking at what we can do to take some of the work we've done and the and the basics here to extend that to cover, you know, storage attached functionality in the OCP configuration.

Philip Alsop

I won't say it's a problem, because it's probably a nice thing to have, but do you find yourself sometimes wondering what you should be doing? Where do you draw the boundaries? Where should you get involved, where shouldn't you? Or do you think you've got a fairly, because it seems to me these days storage, as you say, and then you've got the servers and then you've got the whole data, the storage. I get that, but the data that sits on it is becoming important, which is a slightly separate. Obviously it's not unrelated. So yeah, if that makes some kind of sense.

Richelle Ahlvers

It does. Is it easy to? The question is kind of where do you draw the line between what one organization does and what another one does? Well, we addressed that through the Alliance agreements. So let me talk about another set of work we've got going right now that all wraps up into the same space. We've got some work going on around storage fabric management, and we're working with five different organizations right now jointly. So this is work that's going on in the other fabrics Alliance.

Richelle Ahlvers

But we've pulled in folks from DMTF to, on the right-hand side, swordfish for the storage use cases, ofa, because they want to help make sure that this is an open architecture that works across all different fabric types. And then we pulled in both Gen Z and CHL as a couple of different fabric types that are enabled them working within this right now. And so in terms of who does what, in some cases it's the same people just wearing different hats, because I also participate and help develop the Swordfish or the Retro Specification. But when we're doing an effort like this, what we actually do is we go through and say, okay, fabric management for storage, the basics are the same as fabric, it's just fabric management. That's Redfish stuff. So let's go make sure all of that's complete and then we can start layering on the functionality we need for storage.

Richelle Ahlvers

So into those little bits and pieces in. It's not something different these meetings are, and this work is kind of a great way to highlight that. The work is this tiny little intersection between what all the other organizations are bringing in. We're not reinventing anything. We're just saying, okay, here's the little piece that's not done yet. Let's go do that and then feed that back in to anyone of the four organizations that needs it and in terms of the overall commitment to openness.

Philip Alsop

I mean I've been involved in the storage industry on and off a number of years, shall we say, and I know heterogeneous storage was talked about and issues within the stuff and I know since then the whole open movement has gained massive momentum. But do you find it now there is just sort of completely, if you like, universal acceptance of open storage and everyone's working towards it, or is there without naming names, or are some vendors less willing to get involved in complete openness than others? I mean just sort of observations.

Interoperability Labs and Standards Development

Richelle Ahlvers

Yeah. So I think there's a lot of question about how folks should best engage, is the way I put that, because there are a lot of things happening Within the storage space. There's everything from groups that are developing kind of think of it as references. This is the they want to develop a spec, or it's not really a spec. It's more like a recipe book that says this is the kind of configuration we want to put together, and then they're basically need to refer to other people's specifications and they don't really largely develop their own specifications. They still need standards groups and standards bodies to come in and say what is the spec underneath this? There's a lot.

Richelle Ahlvers

Security has a ton of these same kinds of things. There's a tremendous amount of sharing and overlap in security and security use cases that cross the boundaries between what the needs are for servers and storage instrumentation and they're crossing about eight different organizations. Right now. It's very chaotic, but it's also so. When you look at something like that and ask a company you say where do I go plug in? It can be a little overwhelming to figure out where the pieces are. But that's why some of these alliances are really important, because it's not just NIA driving these, it's all the rest of these organizations that are doing that. So I've been talking a lot about the manageability space. But if I just talk about our security, which is also something that's very important on the manageability side but we have work going on in OCP with this open organization.

Richelle Ahlvers

In their no-transcript they really like to say here's what to do and the details you fill in from these different specs. And that's what's been going on there. So they're looking at hey, how do we do attestation? So DMTF has a really low level spec for how to do attestation, management based functionality at a low level. And so then over at SNEO we can look at that and say, well, yes, we care about that, we don't need to define that ourselves. Dmtf has that. We can look at how we build that functionality and we refer to those specs that are being developed in DMTF and in PCIC and in TCG. And so one of our other twigs, the object drive twig that's working on specifications for these more intelligent standalone drives, can just pull those in and we don't have to go and kind of again reinvent any wheels there and it all gets pulled in again under this, this manageability umbrella, where it's not just redfish and swordfish, it's redfish and swordfish and SPDM and a handful of other different specs that all get put together to make a comprehensive solution.

Philip Alsop

Okay, and in terms of, I think you fairly recently opened the storage management initiative. You opened your next generation of labs, so I'm guessing that also contributes to the development of solutions. Oh, I'm assuming that. But tell me how it works. I mean, if you've got some sort of sponsor, vendors that put kid in there, or is it? You specified it and then other people are welcome to come in and plug in their hardware software into what you've got, how does it work?

Richelle Ahlvers

Oh, thank you. So yes, we have. We've spent a bunch of time over the pandemic working on how we optimize some of our programs, and one of those involved taking what we'd had in our tech center in Colorado Springs and basically getting rid of all of the old stuff and focusing entirely on new stuff. So we've just opened the new standard innovation lab, and it is. We have basically almost all new equipment but donated from vendors and are starting to build up again from that. In addition to GSM lab, we also support two other interoperability labs which I'll talk about in a minute.

Richelle Ahlvers

But in terms of getting folks to engage with the storage management lab, we are really quite open and we're trying to work on how we enable some of these new technologies. Historically it's been a great place for companies to come in, bring early prototypes of management software, sometimes even early prototypes of hardware. But we focus on manageability sit down, do interoperability, try things out, see what works and then, as the standards mature, bring in clients, bring in others to kind of work and have a place where folks can play with multiple and interact with multiple different types of vendor equipment without you know, worrying too much about you know about the scope, right, you're all within our lab, you're under our NDA agreements, so anything that happens in there is really just all focused on working on improving everyone's implementations, whether it's client side or vendor side. We also support for folks working with our lab within our lab, that again, that early access to the conformist testing so that could feed in to helping the environments, everyone just able to improve their environments because they can run, you know, as their, as their implementations start to move along. They can. They can run that conformist test again just to see how they're doing Some of the other pieces. Like I mentioned, we're working on trying to expand that.

Richelle Ahlvers

In the ME We've talked about quite a bit. You know we're talking about things like Ethernet attached drives, which is a little bit different than we've had in the past. So how do you bring that into the lab? You know we're working with some vendors to say, ok, if you wanted, we're going to bring drive just you know a couple of drives into the lab. What do you want? What do you need from us, from an infrastructure to be able to support that? Do you need a server to? You know, or you know how are they physically going to show up? You know, are they going to show up in an eBof? You know when, when in the cycle? You know, are you going to give us a standalone drive that we can plug in? So we need, you know, some sort of some sort of you know, space in our rack that we can just lay drives and plug them in. You know some form of infrastructure like that, or are you working with eBof vendors? Are you you know how are we going to bring you guys in to start working on that? That's one of the things we're doing right now because With all that, with the Ethernet drives able to just directly embed swordfish.

Richelle Ahlvers

They're like the complete opposite of big iron right, they're little tiny iron and stuff. We need to figure out how we enable them working in the interoperability lab. The other pieces that I mentioned, and this again, I talked a lot about how we work with everybody else. We really do a lot of work with our Alliance partners to help accelerate everyone's development and not overlap as much as possible Within this new innovations lab. We also have space for the Redfish interoperability lab. They host a rack of equipment within our space, within our lab space, and we have another group within SNEA that we should be announcing their lab space as well. That is just ramping up. So we have, you know, three different labs, basically all within the same footprint, all working on, you know, interoperability testing and you know, early access for our member companies.

Philip Alsop

Okay, and maybe just as we sort of finish, I mean either if there's anything else we haven't covered, but also I mean you hinted there, obviously, that you didn't tell us what but there's another interoperability lab and there's anything else you're on the roadmap that you're able to you know, let us know a bit more about, because I mean it sounds like you're incredibly busy anyway, so I'm not suggesting that you need to keep yourself even more busy, but I mean, what else are you looking at doing, I guess?

Richelle Ahlvers

So one of the big things we're adding in this swordfish and the whole storage management ecosystem is, you know, right now focusing on getting some of the feedback from implementations and incorporating those in. We have some other things on the horizon that are fairly straightforward, that are like just looking at, you know, adding support for NMEMI 2.0, that came out last year. All of the work we've done with mapping everything to date has been on the 1.4 base, so we'll be moving all of that over. So our big functionality is really just a pivot to trying to drive folks to help use the lab that we've spent so much time getting up and running and ready for. We know we have some more new equipment coming in shortly. We'd like to encourage others to engage there and get some more clients engaged. And then the CTP function.

Richelle Ahlvers

We expect to have multiple events during this year.

Richelle Ahlvers

We do everything from what we call Machathons to Testithons to Hackathons.

Richelle Ahlvers

So a Machathon one of the earliest tools we use in developing anything within the Retrofishing Swordfish Ecosystems are just Machups, and what we're doing with the Machups within the programs are really the Machathon is if anybody who wants to just get started can come and we will help them either evaluate the existing Machups they have, take our say what does your system look like, what would you like it to look like?

Richelle Ahlvers

And then, as things move along either if they have full-fledged Machups they want to run through this or even a POC we move them into what we call Testathon and the Testathons again, it's a few hour, one day kind of thing, but it's. We'll take it and we'll help them through walking through running against the CTP testing and say here's where your gaps are, here's where your issues are. You can go off and start working on those. So we're trying to stage these events in to help folks understand where they are in the development cycle, where we can help them accelerate If they have issues, questions about this that are outside the scope of hey, we found this additional property we need, which is great. That stuff can all happen as well as coming through the twig and the Enhancements to the spec. But the storage management initiative is really there to help with all of the rest of these pieces and enabling hopefully helping everyone, to enable their development and accelerate their development as well.

Philip Alsop

Okay, I mean, it's lovely, we've covered a fair amount of ground and I'm really grateful for you spending the time. So thank you very much indeed, thank you, thank you.

Kristin Hauser

It's great talking with you. Thank you for listening. For additional information on the material presented in this podcast, be sure and check out our educational library at sniaorg Slash library.