Note: This page is part of the archive.

Archived Content

In an effort to keep current, the archive contains outdated information that may not reflect current policy or programs.

Information Sharing and Analysis Organization (ISAO) Workshop, July 30, 2015, Part 1 of 3


The Workshop was a one-day event held at San Jose State University. Information obtained through interaction with Government and Industry colleagues in these Workshops was rolled up in a Workshop Readout and presented to the (future) selected Standards Organization. They will use the work towards the development of ISAO Standards.

Audio File Media
Audio file
Audio Gal Description

This is an audio file.


Opening Remarks

MIKE ECHOLS:   Good morning.

ATTENDEES:  Good morning.

MIKE ECHOLS:  Good morning.  What a great group!  But I told some elementary school kids that last week too.  So it's all good.


MIKE ECHOLS:  All right.  First, thank you for coming to this wonderful sunshine-y campus to have this conversation today.  My name is Mike Echols.  I am the director of the Joint Program Management Office at CS&C.  I manage information sharing programs, and we're also looking to create new visions for information sharing and to make them actually come to life.

I believe, at this point, I've gotten to know most of you in the room.  We actually know that we have to throw the football to where the receiver is going to be, and if we throw the football to where he is right now, it will fall to the ground.  All right?  And so I appreciate your participation and your insights and your visions, and this is just a first step to us getting to where we need to be.

Just a couple of things.  Rest rooms to either side.  You can find coffee, caffeine, conversation at the lower level.  At the same time, Room A, Room B, Room C, as you see on the agenda, there are some workshop items for later that mention a room, Room A, Room B, Room C.  All right?

Our goals for the day is we want to continue the conversation that we started.  We started this conversation right after the executive order was signed, but in reality, it started a long time ago.  We have to come up with better mousetraps.  There's not one solution; there's many.  But relative to information sharing, we have to come up with better mousetraps, ways of bringing new players into the game, ways of helping people to protect themselves, which raises all boats.  Okay?

Today, everything that you hear, everything that we do, there's no wasted energy.  The workshops that we've held, the RFIs that've received, all the conversation that I've had, you will see reflected in the conversations.  The executive panel, they have studied the documents.  They have studied the information that's come in.  The conversation it's had won't be just on the stage, but we want participation from the audience.  The agenda should reflect that.  We have time set aside so that if there's something that you don't agree with, something that just sounds a little funny to you, we want to have you participate in the process and to challenge the notion.

I'm very passionate about this because I believe in it.  This is not a government-effort program.  We are moving towards doing something special, and we may not see what those results may look like at this moment, but that's what leadership is.  Right?  And I believe that we have the top leaders participating.

So first thing I'm going to do is bring up Dr. Andy Feinstein to welcome you.  He is the provost for—and vice president of Academic Affairs, chief academic officer for the university, and I want to thank him for having us here because great facility, a lot of smiles as we come on campus, so thank you, sir.



ANDY FEINSTEIN:   Well, good morning.  Good morning, everybody, and welcome to San Jose State University.  You're actually sitting in our new Student Union, opened about 6 months ago.  There's actually—will be another—about a 50 percent component that will be opening in the spring, but it's a little slow now.  But give it time.  In about 3 weeks, we'll have about 33,000 students on campus.  So it will be a little bit busier to get coffee, I think, the next time if you stop by here in the fall, but we're all very, very excited that you're here, and I am very pleased to have so many key representatives from the Department of Homeland Security, information-sharing organizations, and industry with us today.

And as that nice introduction said, I am the provost, although I have a difficult time even telling my parents what a provost does.  They want to know what a provost is, but I am.  I am the chief academic officer.  So I focus on all the academic programs on campus.  I work with the deans.  I work with the faculty, et cetera, and we have about 1,600 faculty on campus.  We have several—several hundred staff members as well, and you've heard we have about 33,000 students.

And we're very excited that you're here today because we do believe that San Jose State University is an ideal partner in efforts to increase cybersecurity.  First, we're located, as you know, in the heart of Silicon Valley, the proximity to many companies concerned about these issues, and I'm sure that many companies are here today.  We're a major source of future workforce talent and have been involved in curriculum development.  We now have one of the largest engineering programs in all of California.  We're close to 7,000 students in our engineering program.

We're also home to the Silicon Valley Big Data and Cybersecurity Center and the Jay Pinson STEM Education Program, and besides the STEM programs, we also have 150 other programs on this campus, ranging everything from the humanities and arts, glass blowing, the sciences, to social science, to hospitality, to nursing.  It's a pretty diverse and broad portfolio of programs that we have here.

And our faculty and students are directly involved in collaborative research that creates best practices to prevent and respond to cybersecurity threats.  We have a cybersecurity cluster on campus, a faculty that is focusing on cybersecurity research and students, and it's not just in STEM disciplines.  We have faculty members in the humanities, and we have faculty members in business on the cybersecurity clusters in the research that we're doing.

We are also focused on creating interest in cybersecurity for the next generation of students.  We host summer boot camps right here in this facility that allows college students to test their cybersecurity skills.  We have hackathons.  We have engagement with some of the major companies in town, and we also work even with middle school.  We work with middle school girls to foster interest in computer science.  In fact, middle school has just recently—were invited to Facebook for a competition showing their skills just earlier this month.

So many of you know my boss, Mo—or President Qayoumi—and he really created a strong foundation on this campus, and he served the National Security Academic Advisory Council.  But when he leaves us in August, the university is still going to remain very committed to our efforts in cybersecurity, and I am going to remain very committed to our efforts in cybersecurity.

I'm looking forward to talking with some of you.  I'm looking forward to growing our relationship.  I'm looking forward to having a wonderful time on our campus today, so please enjoy the rest of your day.  And I think I also saw that there's some dining options here.  If you want the best hole-in-the-wall cheeseburger, right across from my office is Peanuts.  Don't tell my friends about that, but it's a great little hole in the wall.  So enjoy the rest of your day.  Thank you very much for being here again.


MIKE ECHOLS:  Thank you, sir.

Okay.  So our first two speakers today are going to give you read-outs from two previous workshops.  The first one, Neal Pollard from PwC, you may say, "I wasn't invited to that workshop."  That workshop occurred during RSA, and it was a White House effort, but we've included those materials.  And we want to make sure that you have the benefit of those discussions as we move forward.  Next, Roman Danyliw is from Carnegie Mellon, is going to do the read-out of the Boston meeting.  So we'll have Neal first, and then Roman will come up.  And then we'll have an opportunity to have a question-and-answer session with both of these individuals.


Read-Out:  April 20 RSA ISAO Workshop

NEAL POLLARD:  Hi, everyone.  My name is Neal Pollard.  I'm a director at PricewaterhouseCoopers Cybercrime and Breach Response Practice.  I am the lead cybercrime person in our New York City office, so kind of bored up there.

We held a session coinciding with RSA in April where we wanted to come out of that session with an understanding of what worked.  We wanted to produce a paper that Mike mentioned, which is on our website—and I think it's in your packet—of suggestions, recommendations, ideas, a gift from the Good Idea Fairy, for DHS and others forming ISAOs and the standards organization especially that's being set up of—based on models of what's worked in the past.  What should be kept in mind?  What will make these ISAOs work?  What's worked in the past for the different models, and what should be baked into the standards and the blueprint going forward as suggestions that will maximize the chances.  If these things will take root, they will last for a while.  They'll scale, and most importantly, they will continue to deliver ongoing value to the members that comprise these things.  So it won't be sharing for the sake of information sharing.  It will be sharing for a purpose.

So what we did is we convened a couple panels, some government folks, some industry folks, some academics, some hardcore techies, some policy folks, and we reviewed four or five models of what is generally held to have worked in the past.  And these models range from formalized structures that are industry specific that have been around to decades to ad hoc groups that formed around a common problem, and once the problem went away, then the group went away.  It served its purpose, delivered value, no other reason, so it self-terminated.

Other models that don't share information specifically on cyber threat intelligence but are effective information sharing models for, as I said, achieving the objectives that were set out in the charter of these groups, which means delivering value via the sharing of information.  So we came away from that—and I'll go on to what the models were in a second, and I think we're going to be available for questions, so we can get deeper in this, but we came away with sort of six general senses, issues, things to be kept in mind that in the past has maximized the effectiveness, the scalability, and the longevity of these groups that we considered as models.

So first is the need to take a trust-group approach to membership, requires agreement on membership definition, criteria, role size, rules for joining, rules for participating, and rules for peeling off when either you're no longer contributing value or you're no longer receiving value from it.  Consensus is important here in what the rules are because we saw in past models that the members define what the rules are, and they're pretty good adhering to it.  So that's the first point.

The second point is the governance and related process standards for ISAO must allow for flexibility.  Certainly, when the earlier stages of this conversation has been going on for most of this calendar year—but these things organize around geographic entities, issues, events, other bases for organization, and sometimes the need for the model changes, the need for the group changes, and so the value will change.  And the structure and the rules and the participation and the substance must be flexible to change along with it.  Otherwise, it will become obsolete.

ISAOs must provide timely and reliable cyber intelligence.   This kind of sounds like a no-brainer, but it's worth putting in there.  It has to deliver something of value.  It has to deliver something a little bit more tangible and usable than someone wants to hack your stuff.  You got that.  Not helpful.  What are they going to do?  How do they present themselves on my system?  How can I use this?  How can I use this in such a way that I'm reading about it in an alert and not in the New York Times first?  So it was—and this goes to the governance and structure, not just the substance.  The governance and the structure must maximize the timeliness and information.  Now, the tradeoff there is quality control, vetting, trust, so that you're not sharing, you know, third party really.  You're not sharing someone's information unless they've had a chance to say it's not too sensitive to share.  That all takes time, that takes process, but if it takes too much time and process, the value drops off precipitously.

Information sharing and analysis groups should adopt common vernacular and technical specifications, and the fifth one, ISAOs should be scalable without losing the value of these other characteristics.  These two issues are critical, and we saved the best for the most important because—in a previous life, I was an intelligence officer, and the adage there is that intelligence is only as good as the consumer.  And there are some institutions that are very, very good and very expert and have the capabilities and the knowledge, and the smart consumers know exactly what to do with threat intelligence.  And there are others that are going to need some help.  Those, unfortunately, will represent the bulk of the source of good intelligence and the machinery of the process that will derive value and deliver value to the members.  They will need some help.  So common vernacular, technical specifications, process perspectives, and the ability to scale both horizontally, including internationally to build and enable relationships abroad, and vertically to enable the middle market, all that flexibility has to be in there but in such a way that doesn't compromise trust, doesn't compromise quality, and doesn't compromise usability.

And this will be a challenge, but as we've seen, as we saw in these models, a large New York City bank has more in common with a large pharmaceutical firm than it does with, you know, Pollard's Local Bank of Northwestern South Dakota.  It's those middle market folks that represent the bulk of the targets, especially now since they're less sophisticated in cybersecurity, but they're also going to represent a large part of the source of the data and the participation, and they're going to need some help.  And these ISAOs, if structured right, can provide that help and derive value from their participation.

And then, finally, a topic of great discussion and will undoubtedly be here is what's the role of government.  Government should participate as a peer, not as a director, not as a schoolmaster, but as someone else who both needs intelligence and can provide intelligence.  And I do a lot of breach response.  I see what I derive from an investigation.  I used to work for the intelligence community.  If you take what law enforcement has in terms of knowledge of the threat, if you take what the intelligence community, the foreign intelligence community has, the knowledge of threat, if you take what I and my colleagues collect and companies have in their systems, those are three different data sets.  They represent three different sets of artifacts.  They were collected for three different sets of reasons, for three different sets of purposes, but if you put those three sets together, you've got enormous insight into the badness that's out there.  So government should participate as a peer, to be members, or share information with ISAOs through other means.  They should not be conveners or managers.

So those are some of our takeaways.  Some of those are kind of no-brainers, but we felt they were important enough to keep on there.

The models we looked at, there were four or five models we looked at that were historically deemed as successful.  These are member driven, data driven, event driven, and risk driven.  The member driven are very familiar.  ISACs are a good example.  Industry oriented, member driven, they convened because they have membership.  InfraGard is another example.  They are not convening about an issue or certain data approach, but you sign up, you're a member, go forth.

The automated sharing models are an example of data-driven models.  One example we considered was the CRIS model, which is managed by the Electricity Sector ISAC, where you're a part of this information sharing model because you hang a box in your network, and the box works a certain way, and it generates telemetry, and you use it.  You enrich your sensors, and you're smarter.

The third model that we considered is the event model, and this was one that had a very rich history and a very rich history of success.  The group convenes together around an event, either something bad that happened or something bad that's going to happen.  One that I have personal familiarity with, a lot of our panelists had familiarity with, which was brought up, not only the panel but repeated in discussion.  Do you remember the Conficker worm and the Downadup issue?  The Conficker Working Group is always held up as a good example of an ad hoc, came around together.  The right people came around together to solve a known problem.  The problem went away.  People disbanded, although we still keep in touch, especially next week.

Another example is Y2K.  Can't prove the counterfactual that because of the Y2K effort, nothing happened.  It may not have happened anyway, but what we did demonstrate was the ability of the U.S. government to come around together, bureaucratically well manage a single problem, and then go away.  So there are lots of examples of these event-driven groups.

Another that was held up, which is not cyber, but I thought was very intriguing was the National Transportation Safety Board.  NTSB convenes around a specific issue for the purpose of producing public forensics on what happened for the purpose of making sure that there's safety and it doesn't happen again.  I thought it was a pretty compelling model.

And then the last issue was a risk-oriented issue—or the last model was a risk-oriented model where people convened—researchers especially convened around a new system or a new—like Internet of things to explore what the risks are.  Maybe nothing bad has happened yet, but let's explore the risks.  Let's convene around this certain system or this process that's now out there.

One specific example that was brought up of risk is a group convening around potential risks in smart cars and wired, wired-up smart cars.  So that was a good model that was held up.

Those are the models we explored.  There are a couple, a bundle of issues that are still out there, and I'll just leave my remarks at this, because I think these are issues that are worth taking up by this group but also anyone else trying to improve the odds that these things will work.

One issue that was mentioned is if ISAOs form up and proliferate, how do you aggregate that information?  If there are hundreds of ISAOs out there oriented around any of these different models or others—and that's probably a good thing.  I don't know.  I'm not a value—I'm not going to put a value judgment on it, but if you have increasing amounts of telemetry and increasing amounts of organizations generating threat intelligence, there is a possibility for increasing amounts of noise.

And a fifth model that was brought up was possibly an aggregator of information.  Again, around—you know, if these are all event-driven ISAOs, then this will be an meta-ISAO, an aggregator of risk-specific information derived from these event-driven or industry-driven ISAOs.  It's an issue that I think is worth taking in terms of how do you avoid and maximize—how do you avoid the noise, a better extracted signal from the noise, so the value keeps growing on these things.

A second is the international angle, the cross-border sharing of information.  There are lots of operational and technical challenges for this, and this is not how does one government share with another government or a U.S. bank share with a European bank.  This can be how does a U.S. corporation that operates globally navigate the technical operational security and legal challenges.  I know there's some really sharp experts on the legal issues of this in the room too, so definitely want to get their input, but data privacy regimes in the U.S. and EU and other places are not exactly well aligned right now.  They're kind of divergent from each other, and this could present challenges too in information sharing.

Do you want me to stay up here for the panel?  Okay.

MIKE ECHOLS:  Roman?  And then after Roman speaks, we will have a question-and-answer session.

While they're getting those slides together, just some more information for you.  So the proposal process for the standards organization is closed.  We have more than a handful of proposals, less than a gaggle.  It's a procurement, so we won't have a lot of conversation about it, but what I will say is that great proposals, and whoever ends up being a standards organization will be a respected entity, and we believe that the community will get on board and work with them.  We were very happy with the input that we received from the community.

Read-Out:  June 9 ISAO Workout

ROMAN DANYLIW:  Good morning, everyone.  My name is Roman Danyliw, and I am from the CERT Coordinating Center.  In the short time, what I'd like to do is just give you a sense of some of the conversation—and I stress only some of the conversation—that occurred in Boston and to orient you a little bit, a little picture of what we knew going into Boston and some of the new things we know now.

So, in February, we had the executive order, which provided some foundational elements.  In March, there was a smaller meeting in D.C. where some of the conversation started, and we just heard about some of the activities during RSA.  And then we got additional information about what the standards organization will start looking like with the grant proposal being released in May.  So, with all of that information, we went into the conversation into Boston to begin really trying to understand what are all the different perspectives from all the different stakeholders about what ISAOs should be and perhaps how the standards organization that's going to be making the various guidelines and voluntary standards that the ISAOs will be adhering to, what that will actually, actually look like.

So, in light of that, there is this desire to have different classes of conversation.  So the workshop was organized around three parallel tracks, exploring first how should ISAOs be formed, what the ISAOs should be doing related to analysis, and then from this analysis, how the various sharing should be occurring.  And the conversation was fairly freeform, structured again around those particular kind of topics, and with some panels used to facilitate all that.

And we had a very good turnout.  About 110 folks showed up, running the gambit of kind of industry, academia, and various government participation.

So, in the first track, there is a ready recognition that there is a lot of prior art, frankly, and a number of models can be possible, and the group really explored those.  One of the repeated themes from the conversation with the group is that, first and foremost, the purpose of the ISAOs is to serve their members, and if you recognize that it's a service to kind of memberships, the group quickly realized also that to kind of tackle those challenges, there would need to be first an appreciation for what members need, recognizing that members have different levels of maturity.  And that would particularly—that would actually imply that ISAOs are not going to be one-size-fit-all.  There's going to be need to do all sorts of various tailoring to those organizations, and pulling from that thread, recognizing that there are any number of current information sharing activities happening that the guidance to the standards organization should be primarily kind of twofold.

First, anything that's done with standards or guidelines should be inspired with what's worked before in the past or at least learned from what hasn't worked in the past, and whatever is specified should recognize that a significant amount of flexibility is going to be required, given that there's going to be different charters that the ISAOs have and the members are going to have different levels of maturity.

Thinking through what's going to make the ISAOs successful, an issue that came up quite a lot is workforce capacity, having all the staff that you need with all the right skills and, frankly, a recognition that there is a gap, and how do we get more capacity and capability to really allow some of this analysis to be occurring and some of this information sharing and, frankly, who should be doing some of that capacity building.

Very practically, the group also talked a lot about what are some of the mechanics that need to occur to make some of this information sharing possible.  One of the common things we talked about is just taxonomies, common languages.  So when one speaks to a different organization, there is a common kind of frame of reference that one is talking about, and then almost a community understanding of how data should be handled, so we can extend a trust model beyond, frankly, the individuals that are personally known.

There was a very diverse set of opinions on a lot of other topics that we dived into that there wasn't, frankly, consensus on, which is what are all the different architectures that are possible.  How could we share information?  Is it hub-and-spoke?  Should we have a centralized kind of model?  Should that be the government?  Should something be decentralized, and should there be federation in that decentralized nature?

There was also a recognition.  Again, because there are existing information sharing activities, what are the incentives?  What's going to be new with this new round of activities with ISAOs?  If one participates, what's the new benefits that are going to be accrued?

And then, frankly, the last piece that was discussed also related to incentives, is what participation should government have, regardless of what that architecture would look like, and regardless of what kind of those incentives, what else could the government be adding?

The next parallel track that occurred was focusing on analysis, and this was really tied at the heart of what the standards organization would be focusing on.  One of the key tasks of the SO is specifying what those baseline capabilities will look like, and this working group really explored those things.

The key things that came out of that initial conversation was very much of what you just heard me talk about, what track one was.  There is going to be a lot of diversity in those ISAOs.  They are going to provide lots of different services, and the group, in a sense, kind of struggled with what is the canonical list that should be enumerated because you want to support all of this different flexibility.  And in exploring what the relationship would be to the various standards published by the SO to how one would be recognized as an ISAO, the working group was really clear on one very particular thing, about how success should be measured, and it should be not measured so much by adherence to the various kind of published spec and standards.  It should be measured by whether the members participating in the ISAO got what they needed, whether the membership was really—was served in that regard.

And in thinking through abstractly what those analytical capabilities should be doing, there was real consensus on this idea of you want to get actionable information.  Analysis is the means to an end, and the end is getting actionable—actionable information that will ultimately help organizations to protect themselves.

In talking about what all the different things that would be happening in analysis, the group very much, like the first working group, trying to understand what the model should be realized, that there's a gap in capability, and there was a thread of conversation about how do we get more capacity and who again should be those—the entity providing and fostering some of that training and building that capacity in the workforce.

And very much similar to what the first track found is that there is a feeling that there should be some commonality across organizations, some lexicon or terminology, so unknown parties could work amongst themselves, was a very strong theme that emerged.

So this working group thought really hard also about what guidance to provide to the standards organization, and so this was enumerated in the following way.  First, ISAOs need to have tremendous flexibility in however the SO specifies what those guidelines should be.  It should not really restrict what the members might need, and the members are coming in with any number of contractual, legal, and regulatory requirements.  And whatever the guidelines and standards, again, need to be sensitive to the fact that the members have those constraints, and they shouldn't, frankly, make life more complicated for them.

Tremendous prior art was mentioned as well, and so the idea was the SO is not starting from the very beginning.  There's a lot of community activity that's already happening, and those things should be referenced and really augmented only where might be required.  And a particular threat that also came up was if the heart of what the activity is would be sharing information with others, there's a need to provide the entity that's sharing that information some ability to state how that information is being used, and it was inconclusive into how exactly that should be mandated with technical or, frankly, policy controls.

And the third thread of discussion that occurred in Boston was exploring what might be the technical requirements for the systems that would enable the automated sharing done by the ISAOs and then what would be the capabilities that organizations like ISAOs that would be adopting some of that technology, what they would actually need to realize a set goal.

Very much like the earlier discussions, this thread, which was largely much more technical than the others, realizes there is again going to be tremendous variability by the ISOs and a recognition that information sharing is the means to an end, and so the driver for what gets shared should come directly from what the membership might need.  And then there was no single answer for what should actually be shared.  That again should be driven by the capabilities that are agreed upon in the ISAO.

There was some light discussion about possible architectures, and the consensus was largely that bidirectional would be the best.  Again, lots of different technical things are possible, hub and spoke, the things that are centralized, but what was agreed upon was that just pushing out information, some set of members just sharing information, and others just being in receive mode isn't exactly what was desirable, something where there is a producer received by a consumer, and then that consumer that got those indicators did their own analysis and then shared those results back to the community would, of course, be the ideal arrangement.

The language of sharing also was a big topic of discussion, and the thinking there was, do we need a common format that specified that everyone uses, and that's the basis of all communication?  And in the course of that conversation, what really emerged is, frankly, that compatibility is more important than the canonical format, which is not to say that that format wouldn't be helpful, but there is a recognition that ISAOs are going to have different partners, and frankly, there is no way to make everyone use one particular format.  And there's going to be need for flexibility to, frankly, catch as catch can, to be as flexible as possible and to be as inclusive as possible.  So, again, compatibility was the key thing, rather than driving any particular format, but in talking about what that format might engender, there was kind of a clear consensus around this idea that context, not just the raw technical information, is important, and carrying around guidance on how ultimately an organization might accept and process and use that information was also a key theme.

Trust came up quite a lot, trust in partners, the individuals you're sharing with, and trust in the data that you're actually getting.  And the group came to a specific idea about how to extend the trust tree, so to speak, further than the individuals and the parties you are already collaborating with, but there was a recognition, for example, with trusting the data, having confidence in it, might come from understanding how the analysis was derived.

And in the social kind of dimension of which partners you trust, there was this idea of we need community norms that are widely understood and forced in the community about how things are handled and how things are traded.

And riffing off of the theme that was mentioned in the track one and kind of track two, back to this diversity of what ISAOs are doing, when you get to this idea of automated information sharing and architecture, an element that came up is that there is some need to mediate organizations that might have different capabilities and different kind of capacities with organizations that may be functioning at a different level, because there is a desire to bring organizations with those different capabilities and capacities somehow together, unresolved, of course, how to ultimately realize that.

So a lot of discussion happened that's kind of summarized here.  There's more information that you can get.  So, first and foremost, I would point folks to the Federal Information Notice Registry that just talked about the workshop.  There's some good information there.  I didn't talk very much about how some of the plenary discussions went.  There's audio recordings associated with that.  There's a transcript that you can find at the URL above.

And then, lastly, from the breakout sessions, which were not recorded, and so you're not going to find them in the transcript, there actually is a white paper summarizing in a lot more detail some of the themes that I've talked about.  And you can get a better sense of not just what was, in a sense, talked about, which I enumerated here, but the different perspectives that were highlighted here.  And you can get that white paper by contacting the e-mail address on that ISO engagement website, like you see up there.

Thank you.  I see a number of kind of familiar faces, some new faces as well, and I actually very much look forward to continuing the conversation we had in Boston here, pulling on some of those themes and make them a little more practical as we're gathered here.


MIKE ECHOLS:  Thank you, gentlemen.  That was a great read-out.

First of all, I want to apologize because I don't see your bios in our program.  They should be there.  I'll make sure that everyone has your bios.  But, aside from that, let's get to the question-and-answer session.  If there is a DHS-specific question, I'll jump in, but the floor is open.  Carlos.

CARLOS KIZZEE:  Was there—in the discussion in Boston, was there a theme across all of the groups about what specific role government plays in the context of—one of the things that you last said about a need to kind of mediate organizations of different levels and capabilities and so on, was there a theme that maybe that's like government role or that's specifically not a government role?  And what was the sort of theme about what government does or should do?

ROMAN DANYLIW:  So, in that particular case, Carlos, there was on real consensus about how much the government should be playing or not.  There was conversation across the spectrum of fully centralized to almost not involved at all.  The one place where from the notes, it would appear that there was a consistent thread that government can certainly help with capacity building, but overall, the thinking was that please give the ability to the ISAOs to serve whatever their members need and whatever that might be with flexibility.  And, certainly, pulling the thread then to the standards organization, as things are articulated that should be done by the ISAOs, they need to be cognizant of, frankly, all sorts of capability and capacity that may be occurring and certainly a lot of focus on small and medium business as well as folks that are already formally involved in individual sharing.

DENISE ANDERSON:  Hi.  I had a question too, and this I to Neal.  When you—it was really more along the lines—not necessarily a question but a comment more along the lines of something you said where it's important to find out before it appears in the New York Times.  I would say that I don't care if it gets in the New York Times necessarily, but what is important to know is what does it mean to me and what can I do about it, and I think that's the difference that we miss a lot of times when we're talking about information sharing.

NEAL POLLARD:  I agree with that statement.

MIKE ECHOLS:  [Speaking off mic.]


ATTENDEE:  Sure.  Yeah.  Like Denise said, I think in this room, there's few questions than comments, but just based on the read-outs you did, which sort of summarizes where this conversation has gone to right now, a couple points.  I think we—one of the ways I typify this often is that it's more a matter of mechanics than engineering now.  You go back 5 and 10 and 15 years, and none of this stuff existed, and we're building off from whole cloth.

Now we have the ability to put things together and the work that has been done in ISACs and ISAOs and all sorts of things.  I think one of the caveats that's important to understand in that, though, is that it's a much broader field, and I think one of the risks we have is focusing in on what we've been doing and saying that sums it up when it really doesn’t.  There's a lot of—you know, in governance and law and all sorts of other areas that haven't been intrinsically part of what we're focusing on inside this particular issue, it really is part of it.

And the other comments I wanted to make as well is on proliferation.  We watched the progression from 1998, you know, an ISAC to a set to this period of proliferation.  Mike's comment, I think, in Boston, 200 over the next couple of years that are defined as ISAOs in this conversation, and in reality, in the afternoon, in Roman's session, we were talking about there's really a much larger number.  It really needs to be thousands of nodes of various types.  One of Neal's comments, the Fortune 100 has more in common with each other.  Maybe that should be a group.  We see this already that you mix a lot of size demographics, and some can override the others.  So this proliferation has been a very contentious topic for the last couple of years, and if you look at this as any other commercial market, where something suits everybody, there is the hole-in-wall burger that the provost told us about, and there's McDonald's, and there's everything else.  And you start to see the shape of where this is all going.

And there wasn't a damn question in that anywhere, so sorry.

JAMIE CLARK:  Good morning.  I'm Jamie Clark from the standards group OASIS.  I do have a question.  Where do third-party providers fit into this picture?  Here's what I mean.  In your sort of verbal calm-outs that's coming out of these discussions, there's sort of everybody is either one of two things.  They're either a sharing party, including the government, who is just a big and hopefully compelling sharing party, or they're an ISAO.  They're a thing that helps sharing somehow.

Now, in other similar efforts, like, for example, they roll out HIPAA regulations 15 years ago where the government took a whole bunch of paper transactions and required that they go electronic and applied security and privacy to them, there were some very defined roles for clearinghouses, for business associates, and basically, it was understood that not everybody is a big boy, and not everybody is going to be able to do this themselves or join a group of big boys, and so there was an explicit role in the schema for portals or helper organizations or third-party clearinghouses or portals that will do it for a bunch of smaller sharers or participants who can't do it themselves.  I don't hear a lot about that role, and I assume that might be something different than an ISAO.  There may be sharer.  There may be an ISAO with a rule that says you have to be this high to get on this ride, and then there may be something in the middle that helps them get that high that stays in the middle.  So can you talk a little bit about whether there's any emerging thought on how third-party providers or service providers fit into that?

ROMAN DANYLIW:  I think the conversation that occurred really hasn't precluded the existence of that. Certainly, we have a tendency—and certainly in the read-outs—to use words, just "members," "ISAOs," the "government."  But I know when we spoke of architectures, I specifically said there wasn't consensus.  We talked about all sorts of ways in which third parties could help organizations.  Providing various services to the different named organizations would be kind of possible.  I think that there just isn't a particular idea of what's the right way, and I think the feedback from Boston is there isn't consensus on that.  And right now, they're just really stressing flexibility is the key theme here, and there's a lot of "to be determined" here as well.  And I think that's part of the conversation here today.

NEAL POLLARD:  Yeah.  And I agree.  I haven't seen the consensus if it's out there, but one of the points that was brought up in our panel is participation is a function of the contribution you're going to make, and if third parties do have a contribution to make, then their participation will be defined around what that is, whether it's a source of information, facilitator of information, or by the way, a source of risk.  And, to your point, they don't know.  They didn't know they were a source of risk.  I didn't know as an HVAC provider, I was a source of risk for a retailer.

So helping them, because either they are in the middle market—and this was explicitly brought up earlier.  They are in the middle market, and they're not with the big boys.  And one thing there was consensus around is you need to be scalable horizontally so that if it is oriented around an industry, both providers and the middle market participants of the industry can extract equal value out of it, like the big boys who have been doing this since PCIP and PDD-63.

Also, in some of the risk-oriented models we were talking about, Internet of things, smart cars, I mean, you know, if I'm doing a security assessment and I'm looking at a large store and trying to figure out where to start, I don't instantly go to their HVAC provider as my first step for assessment.  So the complexity as operational technology, consumer technology, and IT started converging together in ways that networkers get really happy about—the security designers, you know, go apoplectic—that's a value of participation right there as just having insight.  It's like, "Did you know you just lashed this to this?"  "Oh, no, I didn't.  We probably should rethink that."  If there is a group that exists to do that for specific systems or specific applications or specific market areas like smart cars or whatever, that's valuable, and it wouldn't be necessary, you know, Dr. Evil is planning an attack through this vector.  It would be the risk oriented, like we're not sure if anyone is going to exploit this risk, but there's risk here.

MERIKE KAEO:  So Merike Kaeo from Farsight Security.  I've been dealing with information sharing issues for probably 10 years, being part of closed groups, and then in the last 3 years discussing more international aspects.  One of the work that's going to be happening very soon as the first group, which is an organization where national and corporate certs basically convene, they're going to start a special interest group dealing specifically with creating a policy framework for information sharing, and this is primarily for legal contracts and governance, so that people know what to share, with whom they're allowed to share, and also what are they allowed to do with the information that is being shared.

And so one of the questions out of this is is this something that has been discussed in the workshops in terms of are you guys also looking at creating policy because it would be really nice if we could have a global policy framework rather than three or four different ones.

MIKE ECHOLS:  Before I get to that question, going back to the last question, we have been contacted by bunches of third-party providers.  The model is not baked.  It's wide open.  We do not want to paint ourselves into a corner, and we're telling the standards organization, whoever that might be, "Don't paint yourself into a corner here.  We want to understand where we need to be able to go in the future."

Back to this question, one of the things that should emerge from all of these conversations and from the conversations that the standards organization is required to have with academia, private sector—public/private sector entities is that policy type of discussions should roll out of that.  We should be able to inform government leaders from these discussions of those specific issues that we need to be looking at.

MARTY LINDNER: Marty Lindner, Carnegie Mellon.  Just to follow up on the third-party piece, in Track 2 in Boston, it was talked about briefly but not very long that there are managed security service providers that are in the business to support an oddball collection of companies basically, and not to go deep into the conversation, but there was a lot of questions about they already have a business model.  They have a way to share to their customers.  Either they do it themselves or they have ways to do it.  And whatever we do here, we don't want to disrupt that because those managed security service providers are a great source of intelligence.

So to your point about third parties, there is a class already that exists.

ATTENDEE:  Mike, what does success look like for DHS once this has been awarded, it's up, it's running, it's working to establish members in the ISAO?  And the second part of that question is, what's next? So we've been talking about information sharing is a means to an end.  What is DHS thinking next that will enable this community to take the next set of steps beyond just coming together and sharing information?

MIKE ECHOLS:  I'm very careful not to create policy on the spot.


MIKE ECHOLS:  But the key here and the reality of it is we know that we need to increase information sharing.  We know that we need to improve and enhance the opportunities for people to protect themselves.

If you look back a few years, you see that we started out trying to protect things under this information protection regime.  It doesn't work in cybersecurity.  All right?  So we went from the protection to the resiliency.  Now we're in an enablement mode.  If you look at the other things that are going on with DHS, we've got automated information sharing activities going on.  We have downloadable resilience reviews that people can do for themselves.  Two years ago, we started working on a framework.  Now we have a framework that people can use to understand that baseline.  So there's a pattern here.  All right?

So the next step is to bring more players to the game, more data points.  How do you get those data points, and what data points do you need?  The data points related to—let's take the HVAC companies.  Right?  So, potentially, there is an organization that has a thousand HVAC members who are not even thinking about cybersecurity.  Right?  So just getting them on board to even think about this, even if it's a third party providing that support to them, creates this new opportunity for us to start putting those data points on every aspect of real-life America, not just critical infrastructure sectors.  We started out with critical infrastructure protection.  It's the key.  Right?  That's why they're critical.  But now we need to expand that.  Right?  There's 300-and-some-million people in America, 23 million small and midsized businesses.  So we're trying to create the opportunity for us to be stronger as a country, not as a particular company, as a potential organization.

This is as a country, and so it's hard sometimes to see the end that you're looking for because we are in a leadership mode.  Right?  When you're in a leadership mode, we're looking at the realities right now.  Cybersecurity issues, the attacks are increasing.  More and more people are recognizing that, "Ah!  I could lose my livelihood."  So we need to present that opportunity.

The end result, if we could see it right now, that would be one of those typical public-private partnerships, government initiatives.  This is not the typical government initiative.  Right?  We can't see the end result yet.  We're defining it.  That's why this is so important.

CHRIS BLASK:  Chris Blask with Webster University and various ISAOs.  That was so good that it's almost enough to shut me up, but that hasn't happened yet.

But that really is the point.  A couple of the comments, Merike's comment, the OASIS comments earlier really speak to this.  We're talking everyone.  My mom, all of us are part of this.  So it's not a dozen ISACs, it's not 200 ISAOs, it's not 2,000 ISAOs.  Every MSSP, but also every vendor, Cisco and Palo Alto and everyone, they are part of this.  They are information sharing knowns.  The ISAOs, when you talk about them, are specific organizations that particularly focus on this.  But most of the information sharing is going to happen by organizations that are doing other things. You know, like the N-Dimension is a great example in the critical infrastructure space, not a big company, but working with utilities and infrastructure, we can give them information and not worry about all the customers they touch.  Lancope, another one, a thousand customers, you don't need to go out and reach them all yourselves.  So it's, I guess, broadening the mind.

The bidirectional comment, while I understand that one—and that's almost—I think it's a metric of some of the mistakes we're making because right—understandable mistakes.  Because we're information sharing people and we want to have bidirectional relationships, the vast majority of people involved are never going to share anything, and that's fine.  So it's broadening the scope, anybody you can think of, the policy issues and how we route all this that Merike is talking about.  These are really the keys.  It's less to do with how we've done things in the past and more to do with how we expand this to be non-exclusive.

FRANK GRIMMELMANN:  Frank Grimmelmann with ACTRA.  Roman, in your comments—over on the other side, Roman, over here.  I didn't want you to get lost there.  In your comments, if I interpreted what you said correctly, you indicated there was pushback on establishing standards per exchange.  Is that in terms of the platform, or is that in terms of a standard?

Having seen this go through health care when the ANSI X12 standards, HL7, and others came into play, and the tremendous cost of being able to get a standard per exchange and with STIX/TAXII as an adoption standard that seems to be emerging within the United States and can certainly be interfaced international standards, why would there not be a desire for a standard of data exchange?

ROMAN DANYLIW:  Thank you.  Yeah.  To refine that a little bit, it's not to say that there isn't a desire for a standard.  The group just stressed compatibility, which is this desire of you don't want to be in the situation where the standard is defined; no sharing will happen unless this standard is used.  And so the thinking was yes, to be—for the optimal arrangement, you know, fast-wire kind of speeds, machine to machine, you are going to clearly need something like that, but you don't want to close your front door to perhaps organizations and partners that aren't there yet or perhaps use something else or perhaps made you something in some raw format.  And so the thinking, again, was have as wide of an aperture as possible was the starting point, not to necessarily preclude that there's a desire to have a standard, and that wouldn't be some enabler for real machine-to-machine conversations.

FRANK GRIMMELMANN:  So it really goes back to the flexibility.


FRANK GRIMMELMANN:  And that makes a lot of sense.


JOHN ABELES:  John Abeles from System 1.  I come from both a theoretical and operational environment.  One of the issues I'm dealing with right now is I've been—

Oh, sorry.  Is this better?  Okay.  Sorry.

I've been approached by a number of organizations that want to start ISAOs.  They're very different models, very different sectors.  One of the problems that they have is they keep saying, "Well, the government wants this," but they're looking at how long it's going to take to get funding for this.

Now, there was supposedly, what, $1.6 billion that was part of funding in DHS that could go out for grants?  Is there an on ramp to get the ISAOs that money, so they have some seed money to really get started?  I know there was some discussion early on about that.  I just don't know what the status of that is and what I can tell some of my constituents.  I mean, I'm working with associations.  I'm working with local governments.  We're settings things up and setting up infrastructures.  So you can touch a lot of people because a lot of this comes down to supply chain.  If you look at what's happened with Target, you look at what's happened across the board, small and medium-sized business, they get bad actors into their systems.  They're uploaded.  You look at some of the things that happened in Europe, in Germany.  There have been major implications of this, and how do we shorten the process that we can help them?  How do we make sure that they can get some money so they can start their ISAOs and at least start a framework and then information exchange?  Because a lot of them have no cyber at all or very little cyber.

And when Mike mentioned the issue about HVAC, that's one of the issues.  It's supply chain.

ROMAN DANYLIW:  Right.  So I'll let Mike punt on whether DHS is going to be providing on-ramp funding, but to kind of give you a sense for the conversation in Boston, there was an appreciation, again, that there isn't a one-size-fits-all, an approach that will work for a larger company versus small and medium may not be appropriate, and there is a need to provide those heterogeneous approaches.  And that's why it's so key that ISAOs have flexibility based on who their membership is to give them exactly kind of what they might need.


ROMAN DANYLIW:  And we also talked, actually at length, at what would be the business model for kind of participating, and we really didn't have a consensus on what the right way is to plan.

JOHN ABELES:  Well, is there actually government money for ISAOs right now that we can tap into?  I mean, I've got a number of constituents who would like to—well, would like to start something.

ROMAN DANYLIW:  I would look to Mike for that.

MIKE ECHOLS:  So the first thing, John, is there's government money because there's $2.2 million per year for standards organization.  So that's sort of our first step.


MIKE ECHOLS:  And getting that up and running was a priority of the White House.

JOHN ABELES:  Agreed.  And I think that's a great—

MIKE ECHOLS:  Relative to we've heard the incentive question, we've heard this issue of expansion and making this an essential part of the cybersecurity activities, and so all of that type of information, which is still kind of coming together from workshops—this is our third workshop.


MIKE ECHOLS:  And we've yet to meet with state, local, tribal, territorial.  And so that information will roll up, and that's what I meant by this information becomes very valuable in future policy.

JOHN ABELES:  Yeah.  I'm just trying—instead of doing things sequentially, I'm trying to bring the concept to market earlier by doing things in parallel that can be done by private sector organizations and then kind of right-size them later, if they need right-sizing.  There are a lot of different models that are going to be harnessed to implement ISAOs based on the environment, based on governance, based on a whole bunch of issues.

MIKE ECHOLS:  Sure.  And that's the kind of information that we take back to the White House and DHS leadership to say, "Here's the opportunity.  There's some inertia here."


MIKE ECHOLS:  "Here's what we need to do to move this along a lot faster."

JOHN ABELES:  Okay.  Well, thank you very much.  I mean, just need to kind of bring those things up the chain.  Thank you.

MICHAEL HAMILTON:  Hi, guys.  Mike Hamilton.  I'm wondering if the conversations have progressed to the point where anyone was talking about sustainment and business models.  It can't be grant money that pays for this forever.  It can't be a group of volunteers.  I hesitate to think that corporations are just going to jump in and do something beneficial for everybody without a return on that investment.  Did the conversation get to that, and if so, what came out of that?

NEAL POLLARD:  This came up briefly, and I think it was actually a side discussion.  The panelists didn't address it head on in our session, and it came up sort of peripheral to your question, but I'll throw it out there.

We didn't address what the role of government is in funding the startups of these.  What we did say is—and this goes to another point—what does a successful ISAO look like?  What is a measure of effectiveness of an ISAO?  And if an ISAO is funded by its members and it keeps going, then that means the members see enough value to keep putting hard cash into the ISAO, and that's a pretty decent measure of success.

There are plenty of examples out there that have been funded by the Federal Government and plenty of examples out there.  Conficker is a good one where they just sort of did it on their own time, and then, of course, the more established information sharing organizations are member funded.  But if people feel that they're getting enough value that they can measure that value in the cash they gave this year, that's a pretty—that's a pretty tangible measure.

ROMAN DANYLIW:  We had an equivalent conversation as well.  If it's member based, members will want to plan, and it's not something foisted on them kind of from the outside.

CHRIS KREBS:  Chris Krebs, Microsoft.  So a comment and a question.  The comment, going to the earlier conversation about what this process looks like, next steps, and where it's going, I certainly appreciate the context in which the statements about how ISAOs and this process should be for everyone, whether it's your grandmother or Joe's Pizza Shop.  But let's also keep in mind, given the scarcity of resources, both people, talent, capabilities, we also should prioritize our efforts and particularly within the Department's purview and their remand, you know, focus on the most critical, stepping out of the sector's focus.  But, you know, criticality, let's prioritize and focus efforts there, and once we achieve whatever measure of success, we can keep going and pushing it out.

So now a question.  Mike, July 10th, the request for comments, Microsoft and a number of other companies and organizations submitted comments back both on the executive order and the grant process.  Do you have any update or kind of initial thematic analysis of the request for comments?  Not the grant, but the comments that DHS sought in the Federal Register.

MIKE ECHOLS:  We're putting those together.  I expected to have those to provide to you for this session.  I did not, but we will get that out to you in short order.  Very helpful, very insightful.

I will tell you that the policymakers, the leaders are reading all of this.

PETER ALLOR:  Pete Allor from IBM.  I guess I get the last or the second questions.  To echo what Chris just spoke to from Microsoft, the limited resources, I think one of the things that we're missing here is if the organizations who want to join an ISAO don't have instant response and security—those are two different things, by the way—then the information sharing is for naught, and adding more organizations to share information doesn't share more information, because we're now at the limit of how do you analyze it.

So the thought here to interject would be maybe we need to think of how you make adjunct groups, sub-elements within those organizations that are already operating, as well as using the marketplace, because the question is how do I get good information, not just some information or a plethora of data, and then how do I enact it?  Because the mention about STIX/TAXII, great.  Do you have products that can take that in?  Do you have a policy to allow that to happen?  Do you actually know what's going on in your environment?  So we're missing the action part because the information isn't worth anything if you can't action it, and that's a problem that we've seen from the early days of the ISACs.  We've experienced it in the first community.  We're all active in how to make that happen, but creating more organizations to give the appearance of data moving doesn't increase security.  So we have to figure out how we're going to enact things and enable small, medium, and large organizations.  This isn't a small/medium problem.  This is an everyone problem.

ATTENDEE:  [Speaking off mic.]

ROMAN DANYLIW:  So that exact observation also came up in Boston where the word "SGO" is a very tricky kind of word, and so the better way to think about it is it's certainly kind of standards, but it's also guidelines, and so where policy kind of comes in is certainly in scope as well.

NEAL POLLARD:  This didn't come up in our session.  It was more this is what we were going to—these were observations of what worked in the past and we are going to provide to whichever organization turns out to be the standards organization.

As a personal observation, though, I think that what we're talking about is going to be more process oriented than technical oriented.  It's going to look more like the NIST cybersecurity standard in terms of broad definitions of what we're talking about as opposed to ISO or COBIT or something like that.  If this were the Army and it weren't cyber, they'd be calling it doctrine.  They wouldn't be calling it standards.  So the basic process and mechanics of how you share, how you do it right, and what the sort of standard—lowercase—of excellence is in doing this sort of thing, but flexible enough so that any unit can go out there and sort of do it itself.

MIKE ECHOLS:  All right.  Thank you, gentlemen.  We really appreciate it.  A good discussion.


MIKE ECHOLS:  Okay.  Next, I want to bring up Mr. Carl Anderson.  He's with HITRUST.  He's the Vice President of Van Scoyoc Association.  He spent more than a dozen years in the Federal Government and on Capitol Hill.  So his comments are an perspective of HITRUST we're looking forward to.


CARL A. ANDERSON:  Good morning, everyone, and thank you, Michael.  Can everybody hear me okay?

How about now?  Yep?  Okay.  Good morning again, everyone.  My name is Carl Anderson, pleasure to be here.  I represent HITRUST Government Affairs, and just to give you a little snapshot, HITRUST is a security and risk management organization with a primary focus on health care.  And in 2007, the Health Information Trust Alliance, or HITRUST, was formed by a group of concerned health care organizations out of the belief that improvements in the state of information security and privacy in the industry are critical to a broad adoption, utilization, and confidence in health care information systems, medical technologies, and electronic exchanges of health information.

Essentially, in 2007, when we first got started, we developed what we call our Common Security Framework, which is a framework that took together a number of the prevailing information standards and put them into one digestible, auditable, scalable framework for health care organizations to then apply and have either a company like HITRUST or third parties assess during them.  So, as we began to grow, our key focuses were to mitigate and aid in the management or risk associated with those organizations in health information, health information as much like financial information, a heightened sense of requirements, and what that framework allows organizations to do is to develop certain security practices around that information.

So, in the more recent few years, HITRUST has evolved into doing a number of different things, not just our Common Security Framework, or our CSF, even though that's what we're best known for.  It's now in its seventh generation.  Every year, we take the data from these assessments and look for the improvements, and right now, we're at 135 different security controls.  And just this year, we added 14 additional privacy-focused security controls that health care organizations can embed into their systems.

I'll get in a little bit more detail about our CSF in just a moment, but another thing we do every year is we put out an annual health information breach and analysis report, and we do a program called CyberRX.  It's now in its second generation, but that's similar to the DHS Cyber Storm, and that is sort of the tabletop exercise from the C-suite down to the security suite, focusing on incident response, and we have different levels of the injects and scenarios that we apply it through.

Our CSF is adopted by approximately 83 percent of hospitals, large hospitals, and 82 percent of health plans, so those would be health insurers.

In the last 3 years, we've done over 23,000 security assessments either through ourselves or with our third-party auditors, companies like PricewaterhouseCoopers, Deloitte Touche, Ernst & Young, and various other smaller, midsized auditing firms.  It's the most widely used framework in the health care industry.

In recent years, we have moved into other areas, one including this information sharing space.  We've found that a number of our companies that were working with us also wanted this capability, so we developed—began to develop in 2011 this information sharing capability.  Now in 2015, we have CyberRX, which is our—excuse me—our CTX, our Cyber Threat XChange, which is our information sharing platform that we provide to members who would like to engage in that through HITRUST.  We're a federally recognized Information Sharing and Analysis Organization, and we have partnerships through CRADA agreements with both DHS and HHS.  And we often work with other government partners such as the Federal Bureau of Investigation and the Secret Service.

Four other key programs that we do, which I'll touch on briefly in a minute, is our CyberVision program, our Cyber Discovery study, which we just began 2 months ago, and our third-party assurance program, which we began earlier this year, which allows health care organizations to leverage their third-party business associates to work with the Common Security Framework, our CSF so that when health care organizations are implementing the CSF, they can also require their third-party business associates to do the same so that they are both working on the same level playing field when identifying security vulnerabilities and doing risk management.  And then also, since 2012, we engage in an annual security conference each year.

So, as we've evolved, we are primarily focusing not just with our Common Security Framework and our third-party assurance program, but we're working with some of our state partners who are beginning to leverage the security framework, such as our CSF for health care organizations.  And that in the state of Texas now, they have a heightened HIPAA requirements, and they are now leveraging the CSF for organizations in Texas.

We also have a cyber insurance development model that we're now using our Common Security Framework to leverage that within cyber insurance carriers, so that they can do their risk assessments on health organizations and use our Common Security Framework to better identify the risk vulnerabilities to health care organizations as they do the underwriting for their cyber insurance policies.  And then also, we're beginning to further solidify the CTX, our CyberVision program, our CyberRx, and our monthly threat briefings that we do with the HHS, the Department of Health and Human Services, FBI, and DHS.

So just to give you kind of an overall broad viewpoint of where we started, which was with our CSF, I just enumerated a number of the different standards that we incorporated into the CSF, and then we—you know, one of those lovely Venn diagrams that everybody likes to see.  But then just this year, as publicly traded companies use the SOC 2 risk assessments and audits, we now are working with the American Institute of CPAs to also leverage our CSF, so that when a publicly traded health care organization are going through their SOC 2 audits, they can leverage the CSF and sort of get two birds with one stone.  It roughly saves organizations about 30 percent in their auditing costs each year.

So what is our cybersecurity Cyber Threat XChange at HITRUST?  HITRUST CTX automates the process of collecting and analyzing cyber threats and distributing actionable indicators in electronically consumable formats.  It's available in multiple subscriptions levels, but our basic subscription, which is free, includes the following features.  It advances intelligence specific to the health care sector.  Intelligence includes feeds from DHS, US-CERT, HHS, and many health care organizations.  It is tracking the top threat indicators observed that are targeting the health care sector.  It has keyword alerting.  It has indicators of compromise specific to the health care organization, and right now, it takes the threats and the IOCs from all these different vectors that are friendly vectors and distills them into the five major SIM formats so that any of our organizations that are using the CTX can directly drop those into their systems.  And right now, we have approximately over a thousand health care organizations actively getting this information through CTX.

So what is our CyberRX?  Again, as I mentioned, it is very similar to the DHS Cyber Storm, and it's the tabletop exercise in conjunction with HHS, DHS, and FBI on occasion to essentially do the tabletop exercise, have different injects, get different organizations involved, beginning to look at the cyber incident response.  Many organizations, we found have never done one of these, and it's very beneficial.

Just next month, we're doing a health plan specific—so that's health insurance companies.  We have 20 different health plans participating in that at the end of the month and CMS and HHS, and that one in particular is going to have C-suite-level involvement, so CEO, COO, general counsel's office, security office, IT, public affairs all collaborating as a part of those 20 organizations participating.  And to date, now in its second generation, we've had over 1,100 different individual participants in these programs.

So what is our HITRUST CyberVision?  We partnered with NSS Labs out of Boston, and this is essentially an application that companies can drop into their system.  It gives them a landscape, but what it's doing is targeting what is out there in the cybersphere, what those threats are beginning to get weaponized and attacking which applications.  So as a CIO, CSO is beginning to learn—make decisions on what to upgrade, what new platforms to invest in, it can tell them or give them sort of an understanding of their landscape plus what the threat landscape is for those particular products, and we found a lot of good feedback on that program.

The other program that we've been doing now for about 4 years are our—or excuse me—2 years is our cyber threat briefings.  These are monthly that we provide through a webex in conjunction with HHS, and approximately now in 2015, we have about 2,700 registrants.  This is important.  We feel that the health care industry is really made up of 430,000 individuals health care entities.  This is from everything from a small one- or two-doctor practice in rural Iowa to your Fortune 10 companies such as UnitedHealthcare, and what this allows them to do is learn what's out there and then perhaps make these practices—put these practices into place on an individual basis.

So our Cyber Discovery study, this is essentially in the last 60 days, we began to develop this.  We'll be rolling out sort of our results in the fall here, but essentially, what this is, we're trying to develop a better understanding of what the actual magnitude, complexity, relationships of the cyber attacks, the commonalities of these targets, and the degrees of cyber threats persisting within the organizations.  And what we're trying to do is determine what threats are out there affecting health care, how broadly are those threats focused, and what the methods and techniques are, and how well the industry is doing to defend against those threats.

The goal is to accurately identify these patterns with persistence, and to date, we have approximately 150 participants who are benefiting from this free—it's a drop-in tool, and then we're also getting the feeds and are able to do the analysis.

What I can share with you primarily at this early stage from the initial feeds is that a lot of what we're picking up is what we often feared.  First, that the threats and exploits are targeting the health care sector are the same that are targeting other sectors, and second, that the information sharing model mainly can be improved by having consumable and actionable intelligence.  This largely plays into the majority resources and willingness to contribute these indicators, such as what Roman and Neal and all of us have been discussing earlier today.

So, in these last number of years, sharing threat intelligence, but then also based on our 23,000 assessments of just the health care industry for the organizations that share these assessments with us, we're learning that the current collection model simply must account for maturity, sophistication, resources, and interest and priority.  What this data needs to be is consumable and actionable, and it needs to be integrated into the environment and the products through the SIM—into the SIM level, and that's why we've developed through our CTX to take that raw data, either from US-CERT, DHS, or our other partners who are contributing those indicators and distill them, do the analysis, and then put them into the different SIM—the five largest SIM providers or SIM systems.

And then we also need to have metrics in order to determine whether we are adequately guarding against these threats, and that's also why our Cyber Discovery study is going to be so valuable to us, and then also what we're able to gain in our Common Security Framework assurance program is that whether these organizations are just doing the blocking and tackling incorrectly or whether they are benefiting or not benefiting from this cyber information sharing, the indicator sharing.

And then given our experience in this space, we thought that providing some benchmarks or hallmarks of what an ISAO should really include are—one is the collection of this data, whether it's through a platform such as our CTX or doing individual studies such as our Cyber Discovery study, the analysis.  I know we've been talking a lot about the collection and dissemination of indicators, but I think also what we need to do is—which is most important is this analysis, and whether that is taking a platform such as our CTX and using the SIM-level drop-ins through our organizations, because what we're finding is that many organizations, they don't even have the capability to do this, so they need somebody to partner with that can do it on their own.  And then the dissemination of that information, that's the various platforms, and then the consumption.  The SIM integration for many companies, that's all they're going to really be able to do in this space.  We find that it is a small minority of those thousand organizations that are actually taking their raw data, being able to identify their own data, and then putting it into either TAXII or take that data through our CTX program and push it out to others.  Plus, it also takes a lot of time, not if they're just able to do it.  It takes quite a bit of time.

And then also the education, best practices, policies, legal, trends, and that's why I think our CTX program or our monthly threat briefings are also beneficial to our member—the organizations that come to us for this advice, and then also our outreach.  We value our partnerships with DHS, HHS, the Federal Bureau of Investigation, and Secret Service, and then our exercises as well.  I think that kind of gives a well-rounded organization that's not just doing the sharing but also doing the analysis, and then also trying to lift the playing field with the old adage of "rising tide will life all boats" here, because for health care organizations, what they're trying to do is ensure that the level of patient care is uninterrupted during the continuum of care, so from the health insurance company to the hospital to the small practice to the urgent care center.

I'm not even sure I left some time for questions, but I don't want to interrupt the break in case we all need to get a little bit more caffeine if we're on East Coast time.

MIKE ECHOLS:  So let's take two or three questions.  As he's going there, let me ask you a question.


MIKE ECHOLS:  How do you balance out the majority versus the inexperienced and "I need this help now"?

CARL A. ANDERSON:  That's a great question, Mike.  Right now, at least with the—I guess our CTX program, our sharing platform, that's why we made it free, so we don't want the inability to have, say, the necessary funds to be a barrier for entry in this space.  If you're a health care organization, you want to start getting this information, even if it's just for your own educational benefit.  That's also why we're doing the SIM integration for the five different platforms.

What we're not doing is really doing this for companies like Anthem, United.  They're very sophisticated.  They have dozens of security official employees, and they are very sophisticated in terms of what they're able to identify and what they're doing.  They do collectively benefit from this, but what we're trying to do is these different programs to lift the maturity.  But I don't know if we're going to ever reach the—all the 430,000 health care organizations, but as more and more health information is put electronically, the two doctor practices all have cell phones now, all have smartphones.  They're all taking pictures.  They're texting with their patients.  They all need to be worried or thinking about this, not just from a HIPAA standpoint, but also from their own patient safety standpoint.

ATTENDEE:  Are you guys thinking about doing any advocacy work on behalf of health care organizations with the manufacturers of health care equipment, especially when we're seeing so many vulnerabilities come up there?

CARL A. ANDERSON:  That's a great question, and I appreciate that.  We do have a working group with several of the medical device manufacturers and the EHRs working to basically come up with a common—not criteria or framework, but a common way for those organizations to not just have to pick up the phone but can kind of share collaboratively on what those devices are experiencing, how EHRs might be handling the different devices, and then integrating into the hospital system.  So we're working with a number of hospitals, a number of EHR vendors, and a number of medical device manufacturers to look at some of these things and what the trends are, because I think even in recent months, there's been a number of reports that that is on the cutting edge.  And we're hopefully going to be releasing something here in the fall or, if not, the winter or spring with that.  Thank you.

Carlos, okay.

CARLOS KIZZEE:  Carl, a question for you.  Putting this in the context now of the executive order and some of the expectations—and I'm kind of really sensitive to that question and the role of Federal Government.  So you talked about how HITRUST has a CRADA relationship with DHS and an agreement relationship with HHS.  I guess FDA as well.  Could you articulate what value are your members receiving from those relationships and then what value are you as an organization providing back to DHS, HHS, FDA?

CARL A. ANDERSON:  Sure.  I'll handle your first one.  I think the value we can provide our organizations is that, as we all know, a CRADA agreement takes a number of weeks.  It's just a necessary evil dealing with the government.  It's not that government is inefficient or anything, but that took some effort to negotiate.  I would hate for DHS to have to go through that also with a thousand organizations.  We can do that.  And then through private user agreements, we work with our organization.  So we have done the sort of heavy lifting, and then our member organizations can agree to that.  So I think based on that relationship, we at least saved them some time working with the government, and then they can partner with us to do that as well.  And then they have access to all our products and services.

The value we're adding to the government, I hope they would certainly view us as adding value.  I think given their participation with some of our workshops, CyberRX now in the second generation, and then also providing feeds to our CTX—I'll provide the example of Anthem.  Anthem is a pretty wide adopter of HITRUST, and what they were able to do when they were first breached is—we all know they self-identified.  They self-reported, but they were able to share those indicators with HITRUST within about 48 hours of doing their due diligence to isolate.  I think their main priority was to make sure their systems were as secure as they could before doing that, but they were able to share with HITRUST and then HITRUST then with US-CERT.  And whether that's the hub that wen t out to others, that was us, but that was a very large attack.  And that provided Anthem, at least in the short term, that anonymity for sharing—getting the information out, making sure that others weren't being attacked, but then also be able to focus not just on sharing the information but then quickly pivot or simultaneously pivot to dealing with their incident response plan.

ATTENDEE:  All right.  For the sake of time, this will be the last question.

PATRICK COUGHLIN:  Hi.  Thanks.  Patrick Coughlin here, TruSTAR.  Quick question.  So how—with such a diversity of players, it sounds like, how do you measure, or what metrics do you use to measure engagement or success?  Is it sheer number of incidents or information shared?  Is it the quality of information shared?  Is it use cases?  And if so, what are some of the trends that you're seeing if you do like cohort analysis of people who join now versus people who join 6 months ago?  Are they willing to share more?  Are they willing to share more sensitive information?  What are some of the metrics you look at when you're evaluating the success of the platform?

CARL A. ANDERSON:  So for just our CTX, I think we're still in sort of the early generation of how this is really working, so it's hard to judge what metric would judge a success.  I hate picking on Anthem, but that's a great example that we can all relate to.  They were handled by an APT.  The way we shared that indicator was as an alert, a heightened alert, so not just your normal everyday traffic that would be in the platform.  With those alerts, the organizations are supposed to get back to us with whether they had seen it on their systems as well within 48 hours, so they were supposed to prioritize that in the queue and then get back to us.

I think one value there was that we were able to, within 48 hours, know that at least the organizations that received the alert and then perhaps some of the others through US-CERT.   I mean, what happened behind the curtain once we gave it to US-CERT, I don't know, but we were able to identify that it was only targeting Anthem and not the other organizations from that one particular indicator or cluster of indicators.

In terms of some of our larger, maybe that goes back to what Michael was asking.  I don't know if we're ever going to reach those 430,000 organizations, but the idea is you get out there.  You have a wide variety and diverse set of platforms and programs that hopefully you have something that everybody can either find a value and use.  And, hopefully, that's how we're to improve this.

I think we're still in the early generations.  Certainly, the information sharing piece is still working out.  I don't think I need to make a plug here for what Congress should be doing here.  That was kind of a disappointing week.  Both houses are going to be going out of session without considering the Senate version, but—thank you.

All right.  Thank you.


MIKE ECHOLS:  Okay.  We're going to take a networking break here, a bio break, whatever issue you want to extend to it, and we want to come back in 20 minutes.  And Executive Panel will be ready to present to you.  The question earlier about the RFIs, this panel has considered those RFIs, and we hope to hear some conversation related to that.