Note: This page is part of the us-cert.gov archive.

Archived Content

In an effort to keep CISA.gov current, the archive contains outdated information that may not reflect current policy or programs.

Information Sharing and Analysis Organization (ISAO) Workshop, July 30, 2015, Part 3 of 3

Description

The Workshop was a one-day event held at San Jose State University. Information obtained through interaction with Government and Industry colleagues in these Workshops was rolled up in a Workshop Readout and presented to the (future) selected Standards Organization. They will use the work towards the development of ISAO Standards.

Audio File Media
Audio file
Audio Gal Description

This is an audio file.

Transcript

Session II

CARLOS KIZZEE:  [In progress]—the executive order.  I wanted to now go over what we're going to do ion this panel and in this particular section about that.  If you're looking at the same slide I am, this defines the concept that it no longer is.  The first plan, the first order of battle never survives the initial contact with the enemy.

We were going to break into panels and then discuss what we're going to discuss now in three separate groups and then come together and do a capstone. For the sake of time, we thought it would be a good idea to just do that as the group.  This or anything we've done today is designed, from our perspective, to be a back-and-forth, sort of a bidirectional conversation about some specific topics.  So we really are looking for input and dialogue, and I'll explain a little bit later on the concept of that.

But, essentially, what we're doing is we're looking at an initial set of baseline considerations.  These are things that existing organizations that have been around and have been sharing threat intelligence, threat information, collaborating together and doing the ISAO thing for a while, the ISAC thing for a while, for years—these are some things that we think about baseline, and when you ask that question, "I want to be an ISAO.  How do I?  What do I, or what are some of the best practices or capabilities that I might need to have?" that's what this portion and this section is intended to actually discuss and maybe tease out.  We wanted to start off with some things that we feel are kind of baseline, but one thing that I want to be very, very clear about, what an ISAO needs to be is what its members are paying for it to be.  What its member require is what the organization's charter and day-to-day functions should be.  An ISAO that exceeds its members capability is probably wasting their money.  An ISAO that does not meet their members' capabilities is probably not earning their keep.  Right?  And so in terms of value proposition may not be around for a while.

So the real focus to what a standard for an ISAO should be from a baseline capability standpoint is what its members require it to be, what does it take in order for an ISAC to do X or an ISAO to do X, and that X being what its members require.

So capturing from the language in the executive order, I specifically—we specifically have focused on seven activities that are in the area of what the executive order talked about.

Now, next slide, slide 3.  Remember, the executive order specifically said that in terms of maximizing effectiveness, which is the title of our panel session, we need to focus on robust information sharing related to risks, cyber risk incidents.  We need deeper and broader networks of information sharing nationally.  We need to develop and adopt automated mechanisms for the sharing of information.

And many of you, I think all of you recognize and realize that those activities, some of those activities are going on already.  Some could be done better.  Some need to be done in the aggregate and collected and enhanced and improved in that way, but we want to make sure that we're leveraging and aligning and optimizing what works already, not necessarily thinking that we're having to start from scratch, so, again, the purpose of this conversation, this dialogue.

So our starting-off point is going to be a set of seven—what we're calling baseline capabilities, and I hope that that is easier to read on your slide than it is on what I have here.  The baseline capabilities, we will be talking through, and when I say "we," I'm not talking about the three of we here.  I'm talking about the 73 of us that are here this afternoon.

Contractual agreement capabilities.  As an ISAO, what capabilities do you need in that area?  What does that mean?  Business processes.  What are some of the common-core baseline capabilities that we should have or that you might need?  Member management.  ISAOs, ISACs function as organizations that support the requirements of their members.  So how do you manage those members from initial identification, outreach, bringing on board, and then meeting their requirements?  Baseline capability four, what are operational capabilities and standard operating procedures that you need to have.  Five, technical tools and capabilities, what sort of is the baseline capability in that area?  Six, privacy and proprietary data handling and minimization protections.  Now, again, we're not talking about what those should be.  We're talking about as an ISAO, what does that mean to you that you need to have some degree of those capabilities to meet your members' requirements?  And then last but not least, the ability for peer and external partnerships to develop and maintain those partnerships.

Our goal here is to feed information back to DHS as DHS has asked us to and to feed information to a standards organization to be named by DHS that from our perspectives, our collective perspectives here, here is what each of those things mean, here's what some of the capabilities that each of those things might be in terms of the common core, like what the—across the board, every organization should consider, what we think that should be.  How might we prioritize some of the activities in each of those areas, and did we get it all?  Is there anything else that we might want to add?

Now, this is the part where I have to call out—DHS has very capable and able people who are taking notes, and the comments and the discussion that we are going to have here, it's going to be captured by them and fed back to answer those, you know, what's the definition, how do we validate, how do we prioritize, and were there any additional things.  So, for those of you who are doing what I generally do, which is taking a snapshot of the slide deck, two things.  One, I'll ask DHS to make these slides available to you, and two, I think that when they do, they should be updated with the information and input that your changes have made.  So that's kind of where we're going here.

Next slide.  The model that we're going to take is to walk through each of those capabilities, what we're calling initial baseline capabilities, just to get discussion going, and we have a panel of people who are going to cover one at a time.  What we'll do—and I'll start, so that we have a model for it, is we'll introduce ourselves.  We'll introduce what that capability is and how very notionally, very initially, we've defined that and going through what we think that that means, and then we're going to open the floor for a few minutes, and we're going to ask us here to say, "Did we get that right?  How might we want to add to that?  How might we want to redefine it?  How might we want to edit it or modify it?" so that we can give to DHS and to the standards organization at least our perspectives on what we're talking about when we say in the executive order that you as an ISAO need to have basic contractual agreement capabilities.  What does that mean?  So that's where we're going for the next hour or so.

I'll kick it off with what I'm asking each of us to do on the panel, and that is to start off by just introducing who I am, what my organization is, and taking 30 seconds or more to tell you about my organization.  I'm Carlos Kizzee.  I am the executive director for the Defense Security Information Exchange, DSIE.  DSIE is the defense industrial base's Information Sharing and Analysis Organization, and like several of the other organizations that have presented today, we have a cooperative research and development agreement with DHS.  Obviously, we partner with the Department of Defense, and we've been in practice for 9 years or so, a little over 9 years.  We have approximately 80 members that are actively sharing threat information and threat intelligence among themselves as well as with DHS and with DoD.

We collaborate.  We don't just share threat intelligence.  We actually collaborate about what that intelligence means, where we think it came from, what it might mean in context, and what we can do to not only mitigate it, but to get better quality of sharing and information.

As an organization, I've been with DSIE for all of about 60 days, and I have worked with them for much longer than that in my previous roles in CIS and in DHS, and I have come to really respect the organization and its members.

Now, our first baseline capability is contractual agreement capabilities, and how we've initially defined that is the ability to develop and negotiate governance agreements like NDAs, tech transfer agreements, IP and licensing agreements, member agreements, things like that.  And we feel that the scope of that basic capability is something that ISAOs need to have in order to negotiate and have agreements with their members in order to enable and foster member-to-member agreements like tech transfers and tech sharing and things like that, that organizations need to have the ability to maintain or to develop or draft with other organizations, with peer organizations, ISAO to ISAO, and also organization to government, like agreements with law enforcement agencies, DHS, sector-specific agencies, and so on.

Now, that's just the initial stab.  I don't want to impress you with the fact that we got that right and that is what that means when it's mentioned in the executive order.  So what I wanted to do is to open up the floor and to at least ask the initial question:  How should we, if this is not a good definition, redefine what should be meant by contractual agreement capabilities?  Again, a standards organization, according to the executive order, is going to put out standards about contractual agreement capabilities.  So what do we want that to mean, and what functions do we want to fall under that?

I'll shut up and open up for input from everybody except Chris Blask, who has talked enough today.

[Laughter.]

CARLOS KIZZEE:  All right.  No.  You can, but you just can't have the first question.

Now, silence means we got it absolutely right, so—

TONI LINENBERGER:  You guys are giving me more credit for coordination than I probably deserve.  I'm Toni Linenberger, and I work for the Bureau of Reclamation, and I'm actually here on behalf of the dams sector.  And I have kind of two pieces I think we need to make sure get captured in this piece and maybe the pieces as we go forward.

So the first piece is we talk about contractual agreement capabilities, and sometimes there's an overlooking of the fact that there are several government entities out there that would like to be consumers of this information, but we don't necessarily have the ability to easily enter into contractual agreements with these type of organizations.  That's probably some of the stuff that you've dealt with in your ISAC because you are dealing with Department of Defense, but how does Bureau of Reclamation get to come play in your sandbox?  Because we can't give you money, but yet there is still a value in having that collaboration and that coordination with those government entities.

And the second part of that is not overlooking some of the relationships that have already been developed in some of the sector-specific agencies.  The dam sector is a great example, and part of the reason that we're here and we want to be involved is we have become a de facto ISAO, and what we would like to do in our mission is to continue to share information throughout the dams.  How do we get information to state and locals?  Yes, there are 25 private sector entities at our table, but how do we get information to all of the other 75,000 dam owners that are out there, and how do we leverage some of those relationships moving forward, so that there isn't an impression of "Oh well, it's going to get to that organization, and then it's going to stop," because it needs to be able to continue further down the chain.

DENISE ANDERSON:  So I just thought of something based on one of the questions that you asked, and that is one term we probably should have in here is "memorandum of understanding," because we do do—and we—and I'll get into introducing myself.  But many of the ISACs do memorandums of understandings with other organizations to share information, and that might take away that harsh-sounding contractual agreement, so we definitely need to include that term.

CARLOS KIZZEE:  No, I think that's good.  I mean, maybe even saying instead of "contractual agreement," it's "contractual and/or governance agreement" or something like that.

Toni, on your second point, that dam sector, man, they're always a problem.  But on your second one, hold that question because I think that the last baseline capability 7, peer and external partnership development capabilities, might cover a little bit of the other part of what you're saying.

FRED HINTERMISTER:  Before we get to the next question, I would just throw in there, one thing that has been useful for us—my name is Fred Hintermister.  I'm with the ES-ISAC in the electric sector.  We found that whenever you can find a parent organization or maybe even your SSA which can lend to some of these capabilities as an enterprise resource, go ahead and do that.  We found it very challenging to do our day job as an ISAC and build the capability maturation at the same time until we realized we could lean on the larger organization behind us for things like this as an enterprise service.  So that's just a practical piece of advice.

ATTENDEE:  [Speaking off mic.]

ATTENDEE:  Carlos, I don't know how you've got these.  I looked at the list of seven, and I don't know where you're putting things, so I don't know if this goes here or this is not an operational enough issue to be included here.  But it looks to me like there's a real pivot there.  You go from governance to much more technical things, but you say it's governance agreements such as—where is there a charter and a statement of purpose and what sort of the rules of the road are?  Is that a different—is that a different one?  If that's the case, I'll just be quiet.  It just seemed like a starting point, and when you went from governance agreements down to the minutia so fast—

CARLOS KIZZEE:  Right.  It is in capability number two.

ATTENDEE:  Okay, all right.  Then I'll just wait.

CARLOS KIZZEE:  Again, it's sort of structured in a weird way because we tried to structure it in the order that they listed in the executive order, contractual agreement capabilities, business processes, that kind of thing.  So we sort of confined ourself to that just so we could sort of align with it.  Dave, good point.

I mean, one thought that I'd ask you, Dave—you can holler out without the mic—I mean, do you think that that's a homogenous enough concern to take the first one and the second one and merge them together?  I'll ask you to think about that, and then when we cover the second one, maybe you could let us know your thoughts on that.

All right.  Sir.

ATTENDEE:  Hey, Carlos.

CARLOS KIZZEE:  Hey.

ATTENDEE:  I would tend to away from some of this wording and maybe make it subtle, make it a BC-1A or something like that.  When you—and it's just an approach.  When you mix the word contractual, you just scared a whole lot of people out.

CARLOS KIZZEE:  Yeah.

ATTENDEE:  When you insert the word "tech transfer" and "IP licensing," you just sent a whole bunch of lawyers running around crazy.

CARLOS KIZZEE:  Which I'm okay with, by the way.

[Laughter.]

ATTENDEE:  I wonder why.

So I would recommend that you talk about an agreement—maybe it's a negotiated agreement, and you leave it at that, and it's primary purpose is membership and purpose and vision, and it's nondisclosure part.  And that's where most of the information sharing organizations—notice I dropped the "A" because I look at it as a lower tier.  Then when you move to sort of an upper tier, you are really going into how you would accomplish based on constituency needs, these additional efforts that require more work.  So, in essence, you get going and operating and achieve purpose, and then if you need higher capabilities, you work those in.

Now, granted, some organizations will jump to BC-1A but not all.  In fact, most won't.

CARLOS KIZZEE:  That's good.  This is all good input, but I appreciate that.

FRED HINTERMISTER:  I mean, just more broadly, just thinking out loud, it feels like the broad capability is the ability to negotiate and to facilitate and codify agreements, without the scare terms.

CARLOS KIZZEE:  And leaving out the "such as."

Any other questions?

DENISE ANDERSON:  Yeah, there's two.

CARLOS KIZZEE:  Sir.

DENISE ANDERSON:  Well, Chris has got the mic.

ATTENDEE:  Some guy.  I don't know.

CARLOS KIZZEE:  Sorry.  I should have known.

ATTENDEE:  I just wanted to make the point this is all about policy, right, which is the big issue.  Merike has probably said this in one or two of the sessions, but she's starting a working group at first.  Tom Millar is working on TLP, another working group to expand all that, but, you know, absolutely yes.  There are legal issues and agreement issues and various other things.  We need to enact those things, but I think it's important for everybody to understand that we haven't figured out policy—

CARLOS KIZZEE:  Right.

ATTENDEE:  —to a large extent.  As the ICS-ISAC, we went through this with you, Carlos, when you were at DHS.  So TLP means you can give us information, but our members are Cisco and Palo Alto and so forth.  Can we give it to them?  Can they give it to their clients?  And this is—I think this is an issue that is going to take a lot of people's thought and time in this period, as we go from a sort of star topology to a rich mesh—and ends up being enacted in paperwork, but yes.

CARLOS KIZZEE:  And so we're going to come a little bit more to data handling and some things like that where the ability—and remember what we're talking about here basically with this first one is—and I think that, Fred, put it really well.  It's just basically the ability to form yourself as an organization, to kind of establish yourself and to be able to connect with others.  That is a common-core baseline capability.  Now, what exactly does that mean?  It may not mean that you need to be able to contract.  It may mean that you just need to be able to do MOAs or NDAs or something like that.  So it's, again, what should that standards organization think about when they're doing this?

There was a question here, sir.

ATTENDEE:  [Speaking off mic.]

CARLOS KIZZEE:  Yeah.  And I don't know if everybody heard.  Did everybody hear that?  So he brought out the point about—important and implicit I think in everything we're talking about today is being able to, at some point, answer that question about, you know, "Who says that you met this?"  There's one thing to develop a standard and to say here is what at a baseline you need to be able to do, but then there's another to evaluate did you do it or not.  And that's something that I think has to be decided in the future in the implementation of the executive order, but that's a good point.

So I am ready, unless there's another question, to hand this off to my friend, Denise, so that she can introduce herself, her organization, and walk you through baseline capability number two, but one additional question.  Sorry, sir.

ATTENDEE:  Yeah.  Before moving off, I know we do not use a member-to-member because there's no reason to and need to.  So in streamlining it, that's something you may want to consider. I'm aware of a number of other sharing organizations, same thing.  They are not using member-to-member because it's all handled more efficiently through the sharing organization.

CARLOS KIZZEE:  And I will say the only reason that I included sort of the possibility of there being agreements below the organization is some organizations, when I was in DHS we dealt with, have tiers, and there may be different sharing based on what tier you fit in.  And then one organization had an interesting set of issues about being able to share tech between one organization and another, and actually, the query came up:  Is there a standard template for a tech transfer agreement, because we have  CRADA with DHS, that maybe we could have to satisfy the transaction of technology from company to company?  So that's why I included that there, but that's a really good point.  Most might not need that.

ATTENDEE:  [Speaking off mic.]

CARLOS KIZZEE:  Yeah.  Member being member company, yeah.

ATTENDEE:  [Speaking off mic.]

CARLOS KIZZEE:  And let me ask because I think it's helpful for those two points.  Let me ask.  For the guys that are capturing this, have you captured enough to be able to articulate both of those points?  If you have, then we're good, because we don't have to solve that here.  I just want to make sure that we capture it.

SCOTT ALGEIER:  So this is Scott Algeier with the Information Technology ISAC.  Hey, I think that this is one of those areas that we should mark down as first do no harm because there are multiple models out there that can work, and they—different models work for different organizations.  So when we come up to this contractual agreements and capabilities, we should—when we're working with a standards organization, this is one of those areas that needs to be recognized that there's more than one way to have a contract in place among members and between members that works for that organization. 

And just speaking for my own experience, if I have to go back and change my member agreements to meet some standards, that's a very expensive, costly, time-consuming process for me to get all of the attorneys from all of my member companies to agree on a new member agreement.  So I just want to mark this as one of those areas to be first do no harm.

DENISE ANDERSON:  Yeah.  Going back to that point of you don't want to screw up, it works, obviously the ISACs have solved this problem working with their membership, so—

ATTENDEE:  A question on your implementation test, which is kind of a standard.  Would that mean then that, as someone suggested, that you would follow the incident response frameworks, and if so, then you would follow the global, probably like first and saw the national C-CERTS are working there?  I mean, if you're going to tie it in a national standard, I would go that way; in other words, how an organization operates. I'm throwing that test out there because I don't necessarily believe that's the way we should operate, but someone was saying, "Well, later on, we'll talk about implementation and the standard of care, basically, of being able to operate," and if that's the case, then I'm going to propose now that if you're going to say that, then I would suggest that you have to go with something that's internationally recognized where people actually do incident response, which includes sharing.

CARLOS KIZZEE:  And I think where that comes in, Pete, is standards—that's information that needs to be captured for the standards organization.  Clear—I think what I've got so far—and I'll hand this on to Denise—as a baseline capability, we agree that this basic capability of governance or agreement is important.  Definition is ability to negotiate or facilitate or codify agreements to carry out our business.

We looked at the scope, organization and member, but question member to member, not every organization is going to need that.  That's actually—in my years at DHS, that only came up once, but it did come up.

That's the last that I have on this one.  I will ask that if there's anything that you feel that you need to add to that, capture that and pass it back to DHS because I don't want just this short conversation to be the only opportunity to have input on that.

With that, I'll hand it over to Denise for the second capability.

DENISE ANDERSON:  Okay.  So I'm Denise Anderson.  I've got three hats on right now.  So I'm chair of the National Council of ISACs, and with my three hats, I actually have a very unique perspective.  So I see across all the ISACs, and I see the similarities that we all share—and the differences.  And then, as someone who was employee number two as FS-ISAC and helped grow the ISAC to where it is today, I'm an executive of that ISAC.  I had the unique experience of taking an ISAC that was very small at the time.  We had 68 members when we started in 1999, and we have over—you know, close to 6,000 today.  And staff, there were for a long time, two of us and then maybe three of us and five of us, and now we have 72.  So I've seen that growth intimately within the ISAC, and then I'm also now executive director of the National Health ISAC, and I'm hoping to take this ISAC to the level of FS-ISAC and its recognition today, and certainly, that capability and that constituency is there.

So it's given me a very unique perspective in the unique needs that can be leveraged across the sectors, but also the unique needs that cannot be leveraged within these sectors.  Again, as it's been stressed over time and time again, everyone has to meet the needs of their members.  That's what makes them successful, and the marketplace is going to determine that because either you have members join you or they don't.  So that's the bottom line.

So, basically, with the business process capabilities, the definition there was organizational scoping, establishing the organization, and then what kind of funding you're going to need to do this.  Now, funding is a very key component, and any ISAC can tell you that that's something that they've struggled with as they've grown.  FS-ISAC experienced years where they were in the black—I'm sorry—in the red.  You know, that's not a problem today, but other ISACs that are starting up, it certainly is an issue.  So it's a very key fundamental piece to standing up an organization and keeping it going.

As far as scoping and establishment, obviously there's certain constructs that have to be set up.  Scoping—and I think some of these bullets here capture that.  Defining who your target membership is going to be is absolutely key.  You have to have criteria because, quickly, that will go into a very gray area.  Any ISAC can tell you that.

I know there was a question that came up earlier today about how many ISACs someone can join.  I want to point out General Electric, I think they belong to about seven different ISACs, and that's their choice.  They found unique needs within each one of those constituencies.  Developing your organizational model, what is your structure going to look like?  What—you have your members coming in.  How are you going to serve them?  How are they going to be governed?  So are you going to have a board?  How do you get your initial seed funding?  So how are you going to start off?  And many ISACs have different models for funding.  That's very obvious.  Some of them have member-driven models, member-dues-driven models, but many of them have other unique models, so that's something to determine.

Then sustainability, of course, falls right in line with that.  Once you get the seed funding, that quickly evaporates, and how do you keep that going?  And then how do you develop your relationships with other key partners?  I skipped over scope too, and I think scope is very important because you can quickly get out of scope.  And I think as you're starting up, it's very important to be within scope because you could boil the ocean and not get anywhere.

So those were some of the bullet points we teased out.  I'll leave it out now to the room itself to start.  Yes, Marty.

MARTY LINDNER:  [Speaking off mic.]

There's a class of organizations now that exist that target membership as anybody.  Right?

DENISE ANDERSON:  Is what?

MARTY LINDNER:  Is anybody.

DENISE ANDERSON:  Yeah.

MARTY LINDNER:  Right.  Like I said earlier, the managed security service providers, if you put them in this thing, there's a bunch of things that probably have to change.  Like the scope of the organization talks about who you want your members to be, but there's another question about what are you going to offer to your members.  Right?  Some of them will offer services.  Some of them might offer products.  So I think you need somewhere to cover in there what you will do for your membership.  Right?

DENISE ANDERSON:  Yeah.  You have to determine what products and services you're going to—do we have that somewhere else?  I thought we did.

CARLOS KIZZEE:  We did.  I think under the scope, somebody made the point earlier that, boy, what really defines your scope are the requirements of your members, and it's almost a chicken and an egg there.  Right?  Because who do I exist for?  What am I going to do?  Well, who is my constituency, and what do they need?

DENISE ANDERSON:  So I'll use an example of—so, in finance, the sector, the ISAC formed actually pretty broadly, so they have insurance companies.  They have markets and exchanges.  They have payment processors.  They have the credit cards and the ACH.  They have two groups there, and the traditional banks, communities, institutions, credit unions.  And they've performed an ISAC for all of those groups.

Now, actually, what's kind of happened over time is we've now created special interest groups within the ISACs so that they can have their own little communities, but they'll still under that main ISAC umbrella.

In transportation, it's formed differently, and actually, energy is formed differently.  So energy formed the electric sector ISAC.  They have the downstream natural gas ISAC, and they have the oil and gas ISAC.  So they decided to form around subsectors, and that's how they did that.  So each—but again, that was driven by the unique needs of each of the members.

FRED HINTERMISTER:  Very much so, yeah.  On a practical level, if you're contemplating formation of an ISAO, what I'd really recommend to you is that you consider the scope issue very carefully and avoid scope creep because you will see a lot of shiny lures, and you don't want that fish to bite those lures.  And you can do that by keeping it member driven and requirements driven, so that's very important.

Also, I'm confident that you'll find that your selection of the resourcing model is very important to think through up front because that model will drive your equities as an organization.  It will drive how you relate to other organizations.  It will drive how you develop your own standard operating procedures, techniques and procedures.  It will drive many issues going forward.  So think through that resourcing model and avoid the scope creep would be my practical recommendation.

ATTENDEE:  It appears to me that all of these have to do with the initial development of the ISAO, but there doesn't seem to be really much in terms of ongoing revaluation of capabilities, you know, ability to evolve to join with others as someone.  Is that something that we could consider adding?

CARLOS KIZZEE:  It is, I think—ask that question.  Is it, Wendy? 

ATTENDEE:  [Speaking off mic.]

CARLOS KIZZEE:  Yeah.  Wayne says hello.  The beer ISAO guy.  Yeah.  I was going to capture, and you didn't attorney that.

Fred is going to talk about value enhancement and value assessment, and I think that's where that comes up under the next capability.

DENISE ANDERSON:  But I do think in some cases where you're setting up your governance structure, like your board, you obviously would—and this is where the charter, I think, would come into play.  What is your charter?  What is your mission?  What is your focus as an organization?  And then how are you governing yourself?  So you need to have your board member, a board of directors, and of course, then you have—you set your term limits, your succession rules and all of that, and those would be part of your initial set of documents.

FRED HINTERMISTER:  So not to jump in there too harshly, but if I was a substitute teacher, I'd be hitting the table right now because I look at scoping and resourcing, those decisions up front, as sustainability issues for your organization.  If you don't get them right up front and understand the bright lines and demarcate where you are going to operate, what's your operating space, then you really don't have deliberate intent around sustainability.  So it is a sustainability issue for you.

DENISE ANDERSON:  That's what I was saying.

[Laughter.]

ORLIE YANIV:  Hi.  This is Orlie Yaniv from FireEye.  I have a question about whether we are narrowly thinking of ISAOs as ISACs just not organized by sector, and when the executive order first came out, we were thinking about ISAOs as completely new, innovative, or different ways of sharing information.  So FireEye shares information and collects information from its customer base and refreshes that continuously.  So would we be an ISAO?  We had the impression that we could self-certify as an ISAO, but when I look at this business process, it's not contemplating the single company ISAO where you are sharing information by virtue of your product and service offerings.

CARLOS KIZZEE:  I would say, one, there's a piece of that question that I think DHS should answer.  I don't want to step ahead of on top of Mike in answering that question.  I'll say two things, though.  And I think it was David that brought it out earlier—or John—I'm sorry—that brought it out earlier.  What an ISAO is is defined in statute already, and there is a very, very clear and extensive definition of what an ISAO is. 

The executive order doesn't necessarily call that out and reference the statute that defines it, but it does identify that an ISAO can be public entity, private entity, a mixture of the two, that type of thing.  Some of these business processes, I think, are inherent in a company that's already stood up.  They're sort of like what you'd need to stand up as a company.  I'd also say that this doesn't contemplate not only that sort of single company that wants to define itself in its business processes as an ISAO.  It also doesn't contemplate me as a company wanting to define maybe my relationship with some of my supply chain by directional threat sharing and collaboration in my contractual relationships with them as an ISAO either.  I believe by the definition of ISAO, at least how it's referenced in the executive order, those two probably still fit that category.  But I'd leave it up to DHS, and maybe that's something Mike can capstone at the end.

FRED HINTERMISTER:  Yeah.  I would just add to that a little extra nuanced perspective targeted at perspective vendor ISAOs, and that is not the DHS piece, which I agree with you is DHS prerogative to address, but on the industry side and on the ES-ISAC side in particular, I can tell you that our maturation plans do address stepping up to the vendor community in new ways so that initiatives like the ones that were talked about today can have opportunities to share their information into our ecology.  So we are stepping up to embrace that in our plans.

CARLOS KIZZEE:  That's a good question, Orlie.

ATTENDEE:  So, similar to that, Neal Pollard talked about the event-driven model, and if you have kind of this fine toothiness of what you need, do we risk not allowing that kind of quick ad hoc, highly successful, historically, model to become a part of this?

CARLOS KIZZEE:  Well, that's a very good question.  I mean, I would say that that's definitely a concern because, you know, up to DHS and I guess a standards organization, but given the purposes that robust information sharing related to cyber risks and incidents, deeper broadening networks of information sharing, you would not want to exclude those from that sort of environment of ISAO.

DENISE ANDERSON:  I still think—and sorry to jump in on you, Carlos, but I still think that you still would want to have some of these things.

We formed some working groups.  One of them is the medical device initiative and national health ISAC, and we want to define—we're defining who the target membership is.  We're defining the scope.   So we are doing all those things.  There may not be funding tied to it because it's a short-term project that everyone is agreeing to work on, but you still want—you're still going to scope out many of these things.  And if you say—you check the box and say, "Seed funding, no," you're still looking at it.  Right?  So I think it's still valid.

CARLOS KIZZEE:  I'd add one thing.  Toni is not going to say it, but I would.  I think the dam sector and the organization in the dam sector that she represents is an ISAO, and it's been around for how long?  Ten?  Twenty years or so?

ATTENDEE:  [Speaking off mic.]

CARLOS KIZZEE:  So—and it has gone through most, if not all, of these things.  I mean, it's dealt with some of those issues.  So I think the goal here is the message for the standards organization is to not be so prescriptive, so as to break what we actually need to do, what the purposed of the executive order is to do.

FRED HINTERMISTER:  I agree.  I think we ought to go broad, and we ought to go innovative, and I think we ought to be practical.  And, you know, I do envision a world where ISACs will be stepping up to have cyber loading docks to drop off that kind of information to share.  So I would say keep it broad, not so prescriptive.

ATTENDEE:  Okay.  I just want to go back to something.  You guys keep talking about the statutory definition of ISAO, and there's one major problem with that, which is we're trying to move beyond just the critical sectors with the ISAOs, as I understand, but that's a definitional piece of the statutory part of this.  It says critical sectors in there.

CARLOS KIZZEE:  It does but not exclusively.  ISACs and ISAOs were not defined.  So ISACs were created and implemented as sector-driven ISAOs, but the statutory definition doesn't restrict ISAO to being.  It actually identifies specific functions and activities like gathering data, analyzing data, things like that.  So, functionally, the definition fits more than just sector-based organizations.

ATTENDEE:  Right.  I'm not saying it has to be sector based, but the way it's written, as I recall—and I'm trying to remember.  I don't remember the exact wording.  I know you read it out earlier.  Do you have it?  Okay.  Any formal or informal entity or collaboration created or employed, dah dah dah dah dah, gathering and analyzing critical infrastructure.  And that's the problem, is right at the front, the definition is critical infrastructure.  We've got to—you know, I foresee us running into issues in that space if we continue to define everything off of the statute because especially when we start creating ones that are not governmentally tied at all, we're not limited to that.  But I worry that the government will then in its areas or where it's participating, it may find itself limited by tighter interpretation than we perhaps want it to be.  So just a point for the future consideration.

CARLOS KIZZEE:  Yeah.  I'm definitely a fan, as we've said, of the broader functional interpretation.  I think, though, the definition of critical infrastructure in there that it refers to is also a very broad definition that's not sort of sector based or nationally significant either.  But you're right.  If it limits in any way the common sense application and implementation, we've got to be careful, and that's definitely something we have to capture.

DENISE ANDERSON:  Okay.  Well, I have one more question, and then I'm going to move on to Fred.

ATTENDEE:  [Speaking off mic.]

DENISE ANDERSON:  I think the mic is in the back of the room.  It's coming.

ATTENDEE:  [Speaking off mic.]

Thank you.  I agree Scope is really important, and I understand that the executive order says that this is cyber focused.  But as we're moving forward, I think we also need to be really inclusive in how we're thinking and not get so focused on cyber that we miss the opportunities for greater information sharing across the board.  And again, dam sector is a great example because we cross a lot of lines.  We cross the ES-ISAC.  We cross the oil and gas.  We cross transportation.  We cross water.  We cross energy.  You know, we're across the board, and the more that we can share information, irrespective of if it's just bits and bytes in cyber, the stronger that we're going to be at the end of the day.  So I think we need to keep that in mind when we move forward, and that's probably something that needs to be brought out to DHS, is "Yes, we agree that there is a mission, and that this is supposedly cyber," but at the same time, there's a physical security component to this.  There's an information sharing component to this.  There's a personnel security component to this, and we can't get so focused on the cyber piece that when the physical security piece walks by and says, "Oh.  Well, we're leaving the gates wide open, and that's the data center," we don't say, "Oops, it doesn't say cyber, so we'll just let it keep going."

FRED HINTERMISTER:  Yeah, it's a great point.  You know, as a practical matter at our ISAC, which I guess is a sector-specific example of an ISAO-type organization, we keep it member driven, and our members are very clear with us.  If it's not going to have a perspective operational impact, it's of lower interest.  So this means it could be cyber, it could be physical, it could be hybrid, or it could be something else.  It could be a pandemic.  There's a lot of things it could be.  So we have a broad view on that.

CARLOS KIZZEE:  And I would add my ISAO organization is kind of an interesting conundrum because, specifically, by scope, our scope and charter focuses on cyber threat activity.  We are not the sector ISAC.  We are a cyber organization, a cyber construct within the sector.  So we probably fit a little bit closer to that sort of notional definition of ISAO than we do ISAC.

FRED HINTERMISTER:  So I think we can move on to number three, which is slide 7, and this one focuses on member management operating capabilities, and we've had quite a bit of discussion about keeping it member driven.  One of the first things talked about is engagement.  One of the things I would add is just experiential that we found along the way, is when it comes to engagement, it's all about who you engage with, and it's all about how you choose to engage.  You can actually cultivate a trusted collaborative environment by having thoughtful steps in that regard.  So that's something to really consider.

In terms of recruiting and sustaining your membership, one of the things that we learned along the way, particularly in the recruitment and the on-boarding, is that each sector organization has a membership with a different complexion, a complexion that's very unique.  So you need to get pretty granular about understanding what are the categories of members and what are the drivers behind those categories of members.

One thing about the ISAC or ISAO names is they are really around—the names themselves, around features, but there's the psychology of the sale here because you really want to cultivate engagement to build the data sets that you need through sharing.  And as you build the size, scale, and scope of your data set, you begin to introduce a whole new world of possibilities for the analytic result.  And the end result you're going for isn't the analysis.  It's the meaning that the analysis delivers to operators in your sector—in our case, as an ISAC—or more broadly as an ISAO that answers the member requirements of your membership, whatever that is.  So I would have a lot of sensitivity in your planning around those engagement non-boarding issues.

Two that I'm very fond of are these ones at the end because value delivery is what it's all about.  Value delivery goes to sustainment, which was an earlier question we had.  If you don't have the value delivery piece right, you don't have the sustainment, and why are we all here?  It's moot.  So that's very important to get right.

One of the best ways you can get it right is to measure your outcomes, to understand what your desired outcomes are, the performance outcomes that you desire for your new ISAO organization, and secondly, to have a mechanism, a system of some kind, a feedback loop that gives you continuous improvement opportunities, because I'm confident you'll find that these are innovative organizations.  You're really involved in something that's entrepreneurial if you're setting up an ISAO, and you will learn along the way.  So equip yourselves with the systems that help you to learn, and that's been something that we've learned along the way.

Lastly, I would just say be very imaginative about these things, gleaning the member requirements that inform your strategies as an ISAO.  Be very imaginative.  Be very broad about those.  At ES-ISAC, we just brought in a new senior-level executive, and along with that, we also assembled a C-level team, an advisory group called the Member Advisory Council, and that group will give some visibility to the senior executive and to our organization for resourcing and advocacy, but it will also give a little dose of wisdom from the operator community.

Now, your ISAO might have a different complexion, but I would encourage you to think about how you can set up structures that help equip you for success, and think imaginatively about that.  I think you'll find that beneficial.

DENISE ANDERSON:  I wanted to pick up on one word you said, Fred, and I think it's important because I think a lot of ISACs have discovered this, and it ties in a little bit—so it's definitely member management and membership criteria, but it also ties into funding.  And that is categories of membership.

So we have found that you have to have categories of membership because not every member that comes in is going to want the same things or be able to pay for the same things.  So, for example, a small community institution can't afford to pay what a large global financial company would pay and doesn't want necessarily the same services.  So the ISACs have been able to differentiate the levels of membership by offering special services for each type of category, so that's a strategy there that I think we want to capture because I think that's important.

And the other one is recruitment and marketing, and I think too many times, we're not necessarily thinking in those lines, but when I've been advising a lot of ISACs and ISAOs when they were starting up, you know, what they should have as a baseline to start up—and I absolutely say, "You need a salesperson because you need to have the funding to sustain yourself, and someone in sales or marketing is going to help you do that."  So I don't think we capture that often enough, but that's definitely something that has to be—you know, unless you're going to go with this informal sharing group and that's in your scope or whatever, if you're going to be a self-sustaining organization, that's definitely a concept that you need to be aware of or at least have a strategy for how you're going to recruit and market it.

FRED HINTERMISTER:  I totally agree with Denise.  You know, it's one part excitement, one part education, especially in the early days, and I would encourage you as you think about your categories of members very carefully, like Denise pointed out, think not only of the type of members, but think of the strategic timing and place of your new ISAO development project.  In other words, you might have categories of members at different resourcing levels or pricing levels, but you want to get pretty granular about maybe you have a scaled way that you invite engagement, and you allow them to see the value, allow them to feel a part over time as well as type of member.

CARLOS KIZZEE:  Stacy?

STACY STEVENS:  [Speaking off mic.]

I don't think we would want to have some corporation, you know, like an Enron or something in your ISAO.  So I think that that's a very important aspect of deciding who or what members you want, and again, I don't know if that falls under, you know, assessment.

DENISE ANDERSON:  It falls on there, on board.  I would put it under on-boarding, but we should specifically call it vetting.  I agree with you, Stacy.  That's a good point.

All the ISACs do vet their membership, and it's actually really important for the trust. Each ISAC has their own criteria for vetting their members.  I mean, I could give you what we do, our organizations, but, I mean, each one has a checklist of processes that they use to vet each member that comes on board.

CARLOS KIZZEE:  So for DSIE, our criteria, because we're an industry specific—we're industry specific not necessarily because a threat actor is exclusively targeting defense, but primarily because our sharing model involves trust.  And it's sort of a birds of a feather.  So defense helped us with trust.  So part of our criteria is that you're active in this sector, and then another—the bulk of our criteria is that you're at a level of capability and competence such that you can handle the information without compromise, so the information for you is actionable, so there's value for you.  You're not going to pay for it if it's not.  But, also, you have a degree of mature process, a security processes that can assure everybody else that when they share threat information in the pool, it is not going to go out everywhere through you, because the target of this particular sharing group is not sort of the world should know it, so that we can collaborate and find out what we need to know to be as defensive as well as predictive as we can.  So that's a unique sort of component of ours.

But, again, the point, I think, that I would offer is our criteria is based back to that scope and value.  So we could not have vetting if we didn't have the scope and value is the point that you had made.

DENISE ANDERSON:  Well, the FS-ISAC, I'll tell you they're going to verify that you have a DUNS number.  They're going to verify that you're a regulated financial institution.  They're going to ask for your number, your regulatory number, and they have a certain—other pieces of information, they'll verify the address of the corporation, that the person who is actually applying for it exists.  So they go through a pretty stringent checklist of items to verify that that organization is valid.

FRED HINTERMISTER:  You know, I wouldn't miss the opportunity on vetting either.  Again, to have success at your organization long term, I would ask not only what they are in your vetting questions, but I would also assert what you expect them to do in terms of engagement so they can feel the value and become more attached to your organization over time.

ATTENDEE:  So that goes to the point I was going to raise here and going off of your point on the vetting.  Vetting is a continuous process.  It's not just a one-time process, and that gets to requirements for being a member, and that gets to how are we going to validate that people are meeting their comments, and how am I going to handle dispute resolution, things like that, that will come up between members of my organization over times.  Those are all kind of—I'm not sure if that comes under operational or in here, but those are all things you have to deal with over time because not everything is going to go perfectly from day one.

CARLOS KIZZEE:  That's good.  That's a good capture.  That definitely has to be added, and I think it does belong here.

ATTENDEE:  We utilize vetting very carefully in terms of the collaborative relationship with FBI on any touchpoints, but that's a limited amount of people in each organization.  Companies themselves are defined by being critical to infrastructure, et cetera.

However, some of the really successful sharing organizations that I have observed are communities that are vetting by a member inviting another member only, and it eliminates—what if we didn't have the relationship, for example, with FBI?  And we're seeing that a lot in the European community now where so-and-so trusts so-and-so, and that's where you get in the club.  A lot of the informal organizations within finance, very similar, but I think both models can work, and you may not want to restrict that in the definition as a requirement.

CARLOS KIZZEE:  Definitely.  Almost a hybrid.  I will say one other thing because we talked about sustainability.  The vetting process is also a great opportunity to get to know you, to make sure you know us, that type of thing, but one other question I love in the vetting is who are the top five—who are your top five, maybe, peer organizations that you want to keep up with in a security that you respect and want to keep up with.  Doggone it, if they're not a member of my organization, I'm going to now go and sort of talk to them and recruit them.  But, also, who are the top five entities that you depend on, your supply chain?  Again, if they're not already a member of my organization, if this guy is going to pay dues to me to be a member of my organization, I probably need to go and talk to those guys too and make sure they're just as secure for him.  So that's something that I would just articulate as a practice we have.

DENISE ANDERSON:  Marketing tool.

ATTENDEE:  Carlos?

CARLOS KIZZEE:  Sir.

ATTENDEE:  You just mentioned supply chain.  Right?  I'm interested in this.  Are any of the three of the ISACs that you guys represent considering, in light of this ISAO standup, revisiting how you think about your members and who is in, quote, your ISAC?  I mean, I think that we can all argue that the ISAOs are very well stated, meeting the members' needs, but it sounds as though, historically, they've been more narrowly defined to meeting certain membership criteria.  Given that there will be other opportunities for companies to join other ISACs or ISAOs, are any of the three of your organizations considering sort of relooking at your own individual requirements for membership?

CARLOS KIZZEE:  I can take a first stab at that, and I can say yes.  I can also say we're probably always doing that, and I don't believe that the DHS initiative with the ISAO prompted that.  I think that that's just a function of a changing threat environment and some changing dynamics.

Years and years ago, it made sense to sort of define organizations by trust based on the sector or segment where they work in.  Banking and finance is a really, really good example of how that can get really, really broad.  I mean, it's an organization that has trust over a broad number of entities under that umbrella of banking and finance.

In defense, we are always looking at what that means.  We probably have a perspective that—we are very, very close to aerospace.  In fact, we might share some of the same members in defense and aerospace, but we are seeing more and more of a connection with other sectors as well.  So what does that mean when your name is Defense Security Information Exchange?  We're figuring that out now.

DENISE ANDERSON:  I can give an example of FS-ISAC too.  I mean, 3 years ago, you had to have a domestic U.S. presence to be a member, and the board decided, "You know what?  We're global organizations.  Our presence is all over the world, and cyber has no boundaries, so we need to change our charter and our criteria."  And so we did that 3 years ago, so that the FS-ISAC is now in 38 different countries and has members across the globe.  So it's a constant reevaluation.

I mean, you know, we're not for profits as ISACs, but we also are—you know, we rely on dues for our funding, and so like any business, we have to make sure that we're on top of our game, or else the value proposition is not there.  Someone is going to move on to something else.

FRED HINTERMISTER:  While the microphone is coming, our model is a little different, our resourcing model, so we do not anticipate any change.  We have a very static field of membership and categories of membership.  It's kind of like when you're driving down the road and you see that muddy pickup truck, and it's go the No Fear sticker on the back.

I embrace these other organizations, and we do as an organization.  In fact, we welcome them as collaborative partners, and we're going to lean into collaborative security with them.  So I see these other organizations as potential new sources of that incremental addition to our data set that we didn't have before.  If we're smart, we step up to the table; we work aggressively with them to bring information of value to them as well and help mentor them for success.

CARLOS KIZZEE:  Dave, did we answer your first—

DAVE TURETSKY:  Yeah you did answer my other question.  It was covered very well.

The question I have in this one on boarding, the vetting we were talking about, do ISAOs reject members?  Are the members competitors of those who are in, and is it based on a clear-cut set of criteria and who makes the decision?  And if you don't want to talk about it in the context of an existing ISAC, can you talk about it in the context of what is proposed here?

CARLOS KIZZEE:  I need to make sure I understand, unless somebody else—

DENISE ANDERSON:  Are you saying is it okay to say no to somebody that wants to join you as a member?

DAVE TURETSKY:  Well, I'm both a cybersecurity lawyer and an antitrust lawyer.

CARLOS KIZZEE:  Right.

DAVE TURETSKY:  And whenever you reject somebody who is the competitor of a participant, if it's not based on clear, spelled-out, demonstrable criteria, and if it's disadvantageous to them because they don't otherwise gain the same kind of access to information that could be important for competition, there are issues that arise.  I am just trying to understand what the experience has been so far because I get nervous when I hear, "Well, you have to have a member bring another person in and vouch for them and all that kind of thing," because that's a pretty scary proposition for a competition lawyer, but—

CARLOS KIZZEE:  I would say, so the vetting is to ensure the integrity of the whole, and I am really amazed.  And I've watched this from DHS looking at organizations like these ISACs and then now being in one.  I'm amazed at how much integrity there is in the context of the board of directors, steering committee, and people like that.  They're very, very sincere, and they have to be because they're putting a lot of time and energy into something.

And, also, I'm seeing—it's probably always been this way, but I'm seeing what appears to me to be a lot more contemplation of the fact that I cannot do this alone.  My company cannot do this alone.  At least in my sector, that has not proven to be a problem.  I did come into the job as the executive director with the notion that the vetting process and ensuring the integrity of that process and ensuring the integrity of all of the member management processes, that really belongs to me.  I actually have to be a bit of a watchdog, for in our organization, our board of directors makes the final, you know, yea or nay.  And they make it based on the criteria.  If there's any hint or any evidence of impropriety like, "I don't want them in because they're a chief competitor of mine," then I have to step in and cry foul.  And I will not be employed there for long if that's what the organization is going to do.

But I don't see that as a problem in the organization, but you're right.  We have to make sure that there are mechanisms and processes to make sure that that isn't a problem.

ATTENDEE:  [Speaking off mic.]

CARLOS KIZZEE:  Yeah.

ATTENDEE:  [Speaking off mic.]

CARLOS KIZZEE:  That's good.

ATTENDEE:  [Speaking off mic.]

CARLOS KIZZEE:  That's good.

FRED HINTERMISTER:  That's a great point, David.  Sometimes the structure of the field of membership informs choices made in that regard.  So, for example, in our sector, there are definitive jurisdictions for the power companies.  So the competitive dynamic is not such that it's as big an issue for us, but I can tell you we do say yes to member applications.  We do say no to certain member applications, and we say maybe so to certain member applications.  The ones we say yes to are in Northern Mexico, the Continental United States, and Canada, as well as some other territories of the United States, like Hawaii and Alaska, states and territories, and we'll say no to ones that are not in that footprint.  We'll say maybe so to certain communities of interest like the vendor community.  So what I'm saying by maybe so is we discern in the information sharing space that we introduce to that community, and we're, in fact, engaged in plans to improve the facilitation and cultivation of sharing tailored  specifically to that community.  So there are different actions we take with respect to member queries, but sometimes it's driven by the structure of your field of membership.

ATTENDEE:  [Speaking off mic.]

FRED HINTERMISTER:  That's a great point.

ATTENDEE:  [Speaking off mic.]

CARLOS KIZZEE:  That's good.

ATTENDEE:  Along with member management operating capabilities, along with on-boarding, there should be off-boarding or ways to eject a partner or enforce policy.  You know, somebody goes rogue.  Vendors become too market-y, you know, that sort of thing.

CARLOS KIZZEE:  Good.  Good point.  They are there, but we should have it here.  You're right.

DENISE ANDERSON:  One more and then we'll move on.

CARLOS KIZZEE:  One more, and then we'll move on.

JAMIE CLARK:  Carlos, bad roll of dice.  You got me.   This is Jamie Clark from OASIS.  Let me—

CARLOS KIZZEE:  I'm directing your question to Denise, Jamie.

JAMIE CLARK:  You got it.

Why can't—just to push back on the marketing thing, everything you folks said sounded exactly right for the size of organizations you support and for the service mix you support, which is obviously proven by the market to be something that your component members want.  Why can't there also be ISAO light, which is nothing more than a relatively short mutual agreement, some kind of minor certification or even self-attestation as to capability and an appliance that shares stuff to a database?  Why is that not an ISAO?

CARLOS KIZZEE:  Oh, I would say that there's nothing that you've just said that doesn't necessarily fit here.  We might be talking a little bit too much here about how I do it in defense and how Denise's team does it in banking and finance and so on.  But I would agree with you.  It's, again, what is your organizational scope?  Who are your members, and what does that dynamic require?  And I think I started off by saying that, you know, too much process or too much of this stuff is just as bad as too little.

I am going to move on, but I want to capture your question as we're changing the slide, if that's okay.  There is one other gentleman here who is going to punch me in the mouth if I don't recognize him.

ATTENDEE:  I just want to take the opportunity.  We have three representatives from three different ISAOs—or ISACs up there.

DENISE ANDERSON:  Four.

ATTENDEE:  Sorry.  Four, yeah.

What is your experience with anonymity and anonymization tools, and what does that do, if anything, to complicating things like operating agreements or perhaps maybe streamlining or helping on things like operating agreements?

DENISE ANDERSON:  So that's a perfect segue, isn't it?  Aren't we talking about—

CARLOS KIZZEE:  Very good segue to our next topic.  Do you want to answer that?  And then we can—

DENISE ANDERSON:  Yeah.  So I'm going to tell you.  I'm going to give you an anecdote, and for those of you in Boston, I think I did talk about this there too.  So I'm going to use the example of FS-ISAC, and when FS-ISAC started, we have—and actually, in FS-ISAC, they do have a secure portal, and it does allow for anonymous submissions.  So nobody knows who submits it, and so when people made an anonymous submission, we would pop the champagne cork and drink because it didn't happen very often.  Today, we would be falling down drunk because it's a different problem.

What we did was we actually—there's three things.  We stood up a list server so it allowed for ease of use.  The second thing is that we developed the traffic light protocol, and FS-ISAC actually was the one who developed that, and then others have adopted it since then.  We kind of adopted it from MI5 I London, but we applied it to our sharing structure.  But that gave a very clear guideline as to how information could be shared.  And then we had the trust community.

So what happened was, is that with the list servers, with attribution, the sharing took off, and it was amazing to watch.  So the majority of sharing that happens now within the FS-ISAC is over the list servers with attribution versus the secure portal with non-attribution.  That doesn't say that that exists—that capability doesn't exist.  It does exist, and a lot of times when you'll see members use it, it will be more because there is some sensitivity around it.  But even within the list servers, you will see the mark, TLP red for attribution, and then amber or green for whatever the rest of the information being shared is.

But what it did was people saw who they were sharing with, and it's a very hard concept to grasp.  When I go in and I advise ISACs, you know, "Share with attribution because your sharing will take off," but you immediately see who you're sharing with.  You build the trust relationships because now I know I'm sharing with Carlos instead of some anonymous member out there.  So that was our lesson learned.

CARLOS KIZZEE:  I am going to move on to four and five.  I'm going to cover baseline capabilities four and five together because they are really homogenous, but that is a good translation of that, because capability four talks about operational capabilities and SOPs, you know, just organizational processes for doing what you do, having the processes and the methods and mechanisms.  And then five is the technical tools, and again, I want to encourage that both four and five point directly back to member requirements, you know, this organization is going to do what its members require.

On the scope of anonymity, in DSIE, we don't have anonymity tools internally.  We don't even have the ability to share anonymously.  The organization is built around actually collaboration about the data shared, and so internally, what a member posts, you can see who posted it, and you can go back and talk to them about it.  If you find additional context, you can feed it back to them.  The threat discussion is kind of like the reason for our existence type of thing, so that's quite interesting.

Now, when we share as an organization externally, like with DHS and the CSPI program or something like that, as an organization, we share anonymously.  A member might—you know, we might have somebody that says, "Hey, I don't care if law enforcement gets this or anybody else.  Go ahead and share that," but if there were a query back to us about who did that come from, we'd have to go back to the individual and ask that question.  And, oddly enough, when we go back to the individual and ask that question, a government agency wants to know more about this, "Are you willing to talk to them?"  Guess what?  All 80 companies get to see that somebody is interested in that, and they might want to sort of follow that discussion.  Why is NSA, DHS, DoD asking that question?  So that actually adds to the collaborative discussion that happens.  So that's kind of where we fit on that.  It's a little bit different from the FS-ISAC.

DENISE ANDERSON:  Can I add a point to that, Carlos?  And that one of the beauties of ISACs is that they add the aggregation of sector data and the anonymization—say that fast when you're tired in the afternoon—so that when we're reporting back to, you know, like DHS or whoever, you know, like Carlos said, it's always the ISAC reports.  It's never the organization that reports, and that makes the organization feel a lot more comfortable with sharing, because then they know it can't get traced back to them.

ATTENDEE:  [Speaking off mic.]

DENISE ANDERSON:  From a member, and in FS-ISAC, that is a true statement.

CARLOS KIZZEE:  Yeah, but not in defense.

DENISE ANDERSON:  It is.

CARLOS KIZZEE:  Yeah.  In defense, we will always know.

DENISE ANDERSON:  If a member chooses to do that.  Right?  So if they choose to submit through the secure portal anonymously, we don't know who that is.

ATTENDEE:  So to kind of carry through on that point, because that's a lot of information sharing circles globally, and some of the vendor community, we're talking about this as well, that complete anonymization becomes worthless.  And I say that because of the fact that we need to know how to de-duplicate reports.  So I agree that there has to be an anonymization as in I don't know what assigns any code to the contributing organization, but allows me to rank-order that piece of information over time from that type of organization.  And I need the ability to be able to de-duplicate a report.  So if I got a report out of CSPI and I get 10 like reports, is it circular reporting?  And, therefore, I'm assigning a risk weight to that, and so anonymization of the contributor by name, got it.  But when you get into real information sharing and real analysis, you've got to have that, just like DoD does.

FRED HINTERMISTER:  Well, at the end of the day, it comes down to the mission and requirements of your particular ISAO or ISAC.  We found in our experience that if we follow principles of transparency up front, our information sharing policy is transparent and presented at the outset, it's public, everyone knows what it is.  Our governance policy is transparent.  It' supervisor front.  Everyone knows what it is.  Our personnel code of conduct for our own staff members be included, transparent, up front, everyone knows what it is.  So we're setting the conditions of sharing.  That has been very, very effective for us.

We do offer options for unattributed—entity unattributed sharing, and we do offer options for attributed entity sharing, but one of the things I'll tell you, if you set up an ISAO, depending on your  particular requirements, you may well see the complexion of feelings about this and your field of membership change very interestingly over time.  In fact, Denise was so successful with this.  At first, I remember a time frame when the real fear was, you know, "I'm afraid of sharing with attribution, entity attribution," and then over time, it quickly changed to "I want to share with entity attribution because I want the value coming back to me.  I want specific mitigation coming my way."

So be prepared for that, and I'd recommend at our level here for standards development activity that we keep it high level and not too constraining in that regard to foster innovation.

ATTENDEE:  Yeah, but I don't want to eliminate that because for those of us who are moving to the high end and automation, without having that set of attributes, which I know are being developed in other policy circles, but you have to figure it out in your governance portion, and in some ways, while I get BC for basic—whatever "C" stands for.

FRED HINTERMISTER:  Well, there's entity attribution on submission, and that's one thing.

ATTENDEE:  Yeah, I know, but I was going for—

FRED HINTERMISTER:  That's a thang.

ATTENDEE:  —your high level.

FRED HINTERMISTER:  And then there's—hold on.  And then there's threat attribution on the other side, and that's another thang.  Okay?  So the approach that we're taking for this is to bake it down to a small set of essential elements of information for cybersecurity and physical security along the lines you're discussing, so that there is an economy, an efficiency of sharing going on.  That's our impression.

DENISE ANDERSON:  And with security—

ATTENDEE:  I get what you're saying on the ES-ISAC.  Okay.  I'm talking from a much broader technical aspect.

DENISE ANDERSON:  With security automation, though, I mean, I think the ISACs can still allow for anonymization within automation.  I mean, that's the hub-and-spoke model of the framework.

ATTENDEE:  Exactly.  But when you start out of standards organization, you're coming up with these items.  My point that I was trying to proffer is these basic capabilities are more governance and structural, then you need some operational capabilities to talk about some of these other layers, because I think a standards organization when you look at it has to figure out that there are, in fact, layers—

FRED HINTERMISTER:  Good point.

ATTENDEE:  —to account for the various parts.  So that is what I was trying to get across.

FRED HINTERMISTER:  Good point.

ATTENDEE:  [Speaking off mic.]

ATTENDEE:  [Speaking off mic.]

CARLOS KIZZEE:  I think the point is really well taken that organizations—if I'm standing up an ISAO and I'm worried about scoping and resourcing and sustainability, it's just a really good idea that one of the things I need to be able to do is look at what's out there and that I can leverage, because I do think that there's a little bit of tension between leveraging what's out there and meeting my members' requirements.  But if what's out there meets my members' requirements, then I need to use it.  That's a good point.

DENISE ANDERSON:  Right there.

ATTENDEE:  One observation, and that is FS-ISAC, since you're all within the financial sector, not giving any attribution or context can make sense because it's financial sector.  I think some granularity knowing it's a bank, processing or some other things would help analysts to tie the knots.

An example, when we're going across 14 of the 16 sectors, we need to know at least for the analysts, the context of which sector is under attack, or we can't tie it back to TTP or actors.  So I think it depends on the market you're serving as to how anonymous or a certain level of attribution as an ES-ISAC is necessary.

ATTENDEE:  Okay.  It's been so long now since I thought about it, I don't remember what I was going to say.  Okay.  So the question—or the thing I'm looking at here is I'm hearing a lot of different takes on what we all think this is, and it's what level of guidance needs to come out of this.  The thing that occurs to me might be the thing you want to look at—is set a generalized set of standards for a core concept, and then do two or three tiers of alternate levels of guidance being provided by the standards organization that say, "Hey, if you want to just stand up a bares bone ISAO, you have to meet these two very easy things to do, but if you want to play as a part of a bigger, more integrated piece, here are the standards for that.  Here are the standards for the next level up.  Here are the standards for the next level up," because the problem is what's good for the goose may not be good for the gander in this case.  And I think fin we do this in tiers, let the standards organizations say, "Here's some options.  Choose what pieces of these you want to try and comply to, and the future along you are, the more you can play with others."

CARLOS KIZZEE:  Okay.  I wholeheartedly agree with you.  I think the gentleman right in front of you had a question. 

I'll share, while you're handing over the mic, the use of the term "standards" scares me.

DENISE ANDERSON:  All of us, actually.

ATTENDEE:  I want some standards because let me cross-communication things if people—if it doesn't hurt your area and you're willing to play nice, here's the pre-packaged standards that you can use.

CARLOS KIZZEE:  Right.

ATTENDEE:  They're not enforced, but they're available.

CARLOS KIZZEE:  Right.  So it may be some are near guidelines, best practices, or something like that.  Sir.

ATTENDEE:  Yeah.  I have to disagree with at least the impression I got from what you just said earlier about we should, you know, use the same models to do the types of things we want to do in the future.  I can see trying to extract best practices out of what has been accomplished, but innovation is something that this environment needs more than anything else.  An application of that information is critical to actually be useful.

If we had the answer today, we wouldn't be sitting in this room.  So we cannot get away from the actual encouraging of innovation of how these sharing organizations are going to be valuable to their client base.

CARLOS KIZZEE:  Right.  I think there's two levels to that, actually, and I think I agree with you, finally.  One aspect of the capabilities that the executive order seems to be talking about is—so you want to be an ISAO.  What do you need to have, and what do you need to be able to do?  What are some of the things that you need to think about in order to be an ISAO?  Helping organizations to sort of mature and develop capabilities to do that is very, very important.  We can't be prescriptive, but we do want to give some guidelines and guidance.

But now take two steps back.  This big ecosystem, we want to have it awfully efficient and capable to be able to share at a higher level.  We want ISAOs to be able to talk to ISAOs and members to be able to talk to members and so on, and that's where we want that sort of uber, broad information flow.  So it's almost like we're having a conversation about, you know, different aspects.

We were talking earlier today about the threat information sharing environment.  Now we're talking about what does it take to be an ISAO, and how do we make sure that when we say those standards or guidelines that we're not overly prescriptive?  So I agree with you, I think.

ATTENDEE:  So maybe I'm getting a little too tactical here, and I don't want to push my luck, but there seems to have been sort of a broad embrace of this idea of minimization, and we've even got it there in writing.  And I wonder if minimization as a means to maximizing effectiveness in future cyber threat information sharing is a potentially dangerous bumper sticker to embrace.

In the third group, we talked about really what we're driving towards ultimately, as Rick Howard said, is everybody is sharing everything.  All the good guys are sharing everything, and when we look at minimization, it sort of seems to ignore the idea that there are tools around anonymization or hashing that allow you to share information, protect yourself, and yet still be able to do the correlation needed to deliver real value.

Maybe I'm getting way too tactical in the word choice, but the ideal of minimization is perhaps not something that we should embrace broadly as an organization when we're trying to drive towards everybody sharing broadly.

DENISE ANDERSON:  I just told Carlos that I'm the worst person to talk about the next slide with privacy and data minimization because if I get on my soapbox, I could be here all day.  So I'm probably going to defer.

I agree with you wholeheartedly.  If anybody really looked at how we share information as ISACs and what we're sharing, this wouldn't even be on the table.

CARLOS KIZZEE:  I have to agree too.  It is an issue, though, when we're talking about let's share information to accomplish something, and we're talking to people who haven't been doing it for 8 or 10 years or whatever.  Right?  But we have a lot of privacy concerns over time in different regulatory compliant environments.  We have worked out these processes.  Companies won't share information that we would want minimized because of their liability, and they thought about that in the sharing—in the decision to share.  We thought about it when we put our corporate agreement together for our organization and scope.

So it has its place, but it might not have its place in some of the more established dialogues of sharing.  We looked at DHS in several different advisory committees and groups, the NSTAC and NIAC and at least one other, HVAC, I think—all look at the topic of information sharing at least 10 years ago or over the last 10 years, and they all came out with sort of a common perspective independently.  And that was what was really, really important is that information shared, one, was shared for a reason.  You don't share information to share information.  There is an outcome, and that towards that outcome, the data sharing had to be accurate, so that that outcome could be enabled.  It had to be timely, so that that outcome could be enabled, and it had to be relevant.

And when we looked at what it took to—what those outcomes were and what the data it took to achieve those outcomes that were relevant to us, it was very, very clear that, you know, maybe about 12 years ago in DHS, we were asking for all information that you have, and we'll figure out what we need.  When we focused on that in response to our advisory committee inputs, we realized that we didn't need U.S. person information.  We didn't need a lot of the privacy or even proprietary information.  We didn't always need to know what company it came from.  We needed to know what was the threat, how did it get in, how has somebody mitigated it, and how can we package and communicate that to other people who might be able to catch that before it comes in.  So that was part of a dynamic of the definition—I think NSTAC defined it first—of what information sharing should be.  So that helped shape our thinking.  That's where I come from on that.

FRED HINTERMISTER:  I think Carlos is totally spot on with that.  I totally agree with that.  I mean, we found that from our perspective, every sector is a little different, and every field of membership with an ISAO is going to be a little different.  In our particular sector, our operators couldn't give a flying hoot what the attribution is in most cases, and they couldn't give a flying hoot what the threat attribution is either.  What they're really interested in is information that can be applied to their operating environment.  So it has to be authoritative, timely, relevant, be headed toward an outcome that safe-postures the sector and reduces risk for the entity.  If it does all those things and it's operationally actionable information, that's what they're looking for in our particular sector.  That could be different in some of the other fields of membership.

CARLOS KIZZEE:  And I think the point for having that topic, that discussion, and focus on minimization in the executive order is to kind of make the point that as DHS and the Federal Government are asking for more organizations to play in the sandbox, to play in this space, that those organizations need to have policies that are clear and transparent on that area.  We don't want organizations to make the same mistake that I made in DHS 12 years ago when we were just asking for everything.  "Dump it all in, and we'll protect it.  It won't hurt you.  We promise to not let anything get compromised."

FRED HINTERMISTER:  Kind of think of information sharing like airspace management out here at the airport because you have a layer of sharing that's going on between ISAOs and ISACs in the future here that we're getting ready for.  That's where that minimization comes into play since we go toward automated exchange technologies that enable predictive analytics.

At another level of elevation or altitude, you might have different kinds of sharing, that you wouldn't want to minimize.  You might want to broaden and maximize.

ATTENDEE:  [Speaking off mic.]

So if you are not getting higher levels of elements within NSTACs, it creates a tremendous amount of manual—or in order to leverage the IOCs.  You can say, "These are bad.  That's block them.  That's what the atomic level will do," but on the other hand, do you want to block everything?  So it becomes a real question of operational process around the technology, depending on the level you are in fact sharing.

ATTENDEE:  [Speaking off mic.]

ATTENDEE:  Absolutely.  I think, even with that, you can at least tag the source.  That doesn't create the association.

CARLOS KIZZEE:  Because we are at the 4:30 mark and I know that DHS wants to have a close-out, let me just very, very quickly direct our attention to the last baseline capability.  We've sort of already talked through this, and so it was the ability to develop—identify, establish, and maintain the right types of operational partnerships and relationships.  An ISAO that's forming itself as a regional organization has a lot of touchpoints that it might be able to plug into.  It's got the InfraGard, the relationship with local law enforcement, fusion centers, you know, that type of thing.  So being able to identify where whatever your scope, whatever your members' requirements are, the right relationships that you have to have in order to be able to maintain those relationships, invest in whatever it takes to maintain those.  That's kind of that skill set that is intended by that last slide.

Again, that baseline capability is not intended to be prescriptive to the point of saying everybody has got to have a relationship with InfraGard, everybody has got to have a relationship with their ISAC, or everybody has got to have a relationship with the sector-specific agency.  It merely depends on who you are, where you are, and what's relevant to the outcome of you defending your members.  And I think that's one thing that what DHS and the Federal Government are going with this ISAO initiative that is helpful and beneficial.  It's kind of forcing us to think about our members' requirements and what are we trying to accomplish, and in this area, it's just a relationship necessary to do that.

Now, let me sort of remind us of just one thing about why this discussion, why did we do this today.  We did this because we wanted to be able to, as a group of us here, present to DHS and present to a standards organization, not  set of baseline capabilities and now go and make standards on these capabilities, but to have this discussion and dialogue, have the smart people mining the discussion, your comments and input, package that, and then hand that to DHS and to a standards organization to be named later, so that they have starting off—so they can pull the community together and say, "Okay.  We heard all of this stuff in the workshops and in the different meetings and sessions that DHS has had.  Now let's begin to make sense out of it, and let's do that in a way that does what the executive order says," which is we need to be able to develop some baseline standards on capabilities in these general areas.

So my encouragement to you is, based on what you've seen and what's been presented here, if you have additional input, I would—I will take the initiative and say the goal is to share that information back, whether DHS creates a venue and opportunity to share it between now and when they name the standards organization or when a standards organization gets stood up.  The input, the experience, the knowledge of what we have been doing all of these years must not get lost as we look at new Information Sharing and Analysis Organizations.  So the encouragement is to continue this dialogue and this input when they name a standards organization by your active participation and your active communication with them.

So that's all that I have.  I will hand the floor over to my peers.

DENISE ANDERSON:  You want to say something?

FRED HINTERMISTER:  I am not going to minimize the opportunity to close out the session and enjoy some of that California sun.  Thanks for your time.  Thanks for having us.  Appreciate it.

DENISE ANDERSON:  Ditto.

[Applause.]

MIKE ECHOLS:  All right.  Just a couple of things.  I'm not going to hold you.  First item, in case you did not receive an invitation, I just wanted to make you aware that on Monday, August the 3rd, they are having an AIS session at DHS on Glebe Road, Automated Indicator Sharing, at 1 p.m., and if you need information on that, let me know.  Essentially, based on initiative from the White House, DHS was developed how we would share automated indicator sharing, how we would share between departments and agencies and out to our partners.  They have developed that framework.  They are going to brief that out on Monday, led by Preston Werntz.  If you need more information on that, just let me know.  All right?

And then to close out, if anyone has anything else, here is the opportunity.

[No audible response.]

MIKE ECHOLS:  Okay.  And, lastly, we have an industry location for industry gathering.  Of course, it's an industry gathering.  The place is called—it's called Fahrenheit, and it's on 99 East San Fernando, which is only two or three blocks from here, 99 East Fernando.  It sits on the corner, Fahrenheit.

Thank you for coming.

ATTENDEE:  Mike, thank you.

[Applause.]