Tech Field Day: Security and Segmentation
David Klebanov, Director of Technical Marketing, demonstrates the security landscape available with Viptela’s SD-WAN solution. This includes a breakdown of their segmentation and multi-tenancy protocols, as well as use cases when these would apply.
Recorded at Tech Field Day in Silicon Valley.
Senior infrastructure technologies professional with over 15 years of extensive experience in designing and deploying complex multidisciplinary networking environments.
Tom: Hello, I’m Tom Hollingsworth, and you’re watching Networking Field Day 13. We are here in San Jose, California, with Viptela. We have invited a group of networking bloggers, speakers, podcasters, and luminaries of the community to take part in this discussion, offer their opinions, ask questions and add their voice to the conversation about software-defined wide area networking. If you would like to learn more about Tech Field Day, including how to become a presenter or a delegate, please join us at our website: techfieldday.com.
If you would like to see more videos about this and other exciting technologies, please check out our YouTube channel at youtube.com/techfieldday.
David: Welcome to the next segment. This time, we are going to transition to talk about security. So, we already covered how you add hybrid sites into your environment. We already covered how you enable Brownfield [interpretability], now let’s talk about security. Security is a really key element in here, when – we’re going to talk about cloud security in form of Zscaler, we’ll talk about regional security, we’ll see if we have time to talk about partner connectivity. That’s a little bit more involved.
But security landscape for Viptela has basically, everything we do, as Ramesh mentioned in a previous segment, is predicated on the zero trust principle. So, we identified infrastructure security. What is infrastructure security? Every element – that is vEdge router, VSmart controller, or vManagement system, knows how to protect itself. They all run on the hardened systems. They are all fully protected. There’s no issue with those things being connected directly into the broadband.
Because they run zone-based firewalls inside in order to be able to shield them from the ugliness of the internet. If something tries to hit the device, some sort of denial-of-service attack, we have controls to make sure the device itself is not brought down, so that there’s no meltdown for that device. There’s also controls, because it’s zone-based, to make sure nothing leaks through the device to inside your network.
So, the device is fully capable, and in fact, that’s how people are deploying our devices – they plug them straight into the broadband, right? So infrastructure security is protecting the infrastructure elements themselves. Then of course, there’s the control plane security, which is through a secure communication between the control elements and the vEdge, which is the data plan elements. Then there is the data plane security, which is the [IP Sec], which is the strong encryption that happens between the VEDGEs themselves.
And then, finally there’s the application security, which are the policies where you can say, steer things to your firewall, steer things through IDS, things of that nature. Segmentation is a big use case. A lot of people are after segmentation. It’s, especially, I think it was kicked into high gear after the Target breach. A couple of years ago, everybody eventually found out because the HVAC vendor got access into the other parts of the network where they shouldn’t be accessing.
Segmentation got lots of visibility at that standpoint. But it’s not just about compartmentalizing things. For us, segmentation is, of course, compartmentalization. So, you can have voice and video segmented from sort of PCI or [heap]-sensitive traffic segmented from your guest traffic. So, all of that is segmented off. And when I say segmented, I don’t mean segmented just on the device itself. Our segmentation is carried out throughout the entire network.
So, when I [instantiate] a segment, which you can think about as a VPN, when I instantiate a VPN, that VPN is active on my entire network. So, if I – so the isolation happens not just on an individual device; the isolation happens throughout the entire network. So, my VPN 10 can never communicate with VPN 20, and it doesn’t really matter where those hosts are spread. As long as they are plugging in to VPN 10, they will never be able to communicate with VPN 20 unless I can selectively allow that communication.
So, segmentation is a powerful tool. It’s also used in MNAs and B2B connectivity. Again, it’s a partner use case. I want to onboard a partner, if I just let them into my network, that’s not good, because they’re going to have access to everything. So, I want to put them in a different VPN. And then I’m going to allow communication from that VPN into my production VPNs or my corporate VPNs, but in a controlled fashion.
Maybe I want to send them from VPN 10 to VPN 20, but as they cross from one VPN to another, I want to send them to the firewall for inspection. So, those are things that are possible.
Speaker 2: With your segmentation, you do bandwidth reservations, or placing and quality of service in the VPNs?
David: Yes. So, we can do a full range of QS capabilities – policing, shaping, marking, remarking, trusting, not trusting. So, the full range of QS capabilities is absolutely available. You can also request a certain SLA, as Ramesh mentioned in the last section, you can request an SLA. So, these things could be enacted on an individual device. I can have policing and shaping on the device itself. I can also have a network-wide sort of QS awareness. And I can say, for a specific type of traffic, please only choose a path that is 100 milliseconds of latency, two percent packet loss or better.
That’s a centralized policy decision that is enacted on the network and then you say any circuits that qualifies that, you’re free to take it. If it’s more than one, do active/active. If it’s three, so active/active/active. And keep monitoring it. As conditions change, you want to make sure that – I was a hundred millisecond, now it’s a 110 millisecond – I’m violating the policy, don’t use that connection anymore.
So there’s some decisions that are fully on the device itself, there’s some others that are on the network itself. And then multi-tenancy is another thing, another use-case for VPN. You can think about VPNs as a use case for multi-tenancy, you can have a Tenant 1, Tenant 2, VPN 2. Do I need to make it communicate across tenants? Sure. If I don’t, by default they’re isolated, but if I do, fine, I can allow communication for one to the other, yet insert firewall into this communication.
So, as I’m passing from VPN 1 to VPN 2, go to the firewall, before you’re allowed to hop to the other VPN. So those are the interesting use cases.
Speaker 3: David, can you back up for a minute? What’s the big distinction between segmentation and multi-tenancy? Because in my mind, those are more –
David: Uh, yeah, so you can view it – I think it’s a matter of perspective, the way you want to view your organization. Because at the end of the day, multi-tenancy can be a department in my organization, right? So, multi-tenancy, in the sense of segmentation, depends on what your business is.
Speaker 4: So, one is SPR segmentation for isolation, another one gives you different topologies within each. That’s kind of what he’s getting at.
David: Yeah. So, I know they’re kind of related. They’re related, right? They’re not –
Speaker 5: One’s more isolation, and one’s more policy –
David: Yeah, potato, po-tah-to type of thing. Because at the end of the day, these are the use cases of – at the end, it’s isolation. How to use the isolation is up to you as an organization. Do I just want to isolate guests from PCI, from my HVAC systems, from my IT systems? And that’s one-use case. Or, do I want to treat my HR department, my IT department, my Finance department, and I call them tenants.
At the end of the day, it’s your definition of what it means to be in that segment. Are you a tenant, or are you just a guest Wi-Fi user?
Speaker 4: So maybe you can show that in action, right?
David: Yes. Per-segment topologies what I think Ethan mentioned that earlier. You can build anything. Any topology you can envision, you can build. Based on either security requirements. For example, I may not want full mesh connectivity. I want hub-and-spoke connectivity only. You can do that. Maybe I want something that is really crafted for me. I just want some partial connectivity. I want this side to talk to each other, yet I don’t want the bottom ones to talk to each other. Yes?
Speaker 6: How easy would it be to employ something like a [MPLS] central services model? Where you want people to get to services, but not each other? That’s kind of hub and spoke with a barrier.
David: MPLS services, but not each other. So, if I take –
Speaker 4: It’s almost like a hub and spoke, but you want to only access services on the hub. Yes. I tell you – when you show the servicing session, it will be exactly that. So maybe you should just move on to that.
David: Hold your thought for a second.
Speaker 7: Short answer: it’s very simple. [Laughter]
Speaker 3: It’s just route filtering.
David: So, I can even go as far as – when you say “any”, we can even build point-to-point topologies. So, imagine that if – it kind of sounds like why would I need an SD-WAN for that, but I don’t just build point-to-point topologies – I have all the other options. But I have a VPN, which is, for example, think about a partner use case. I have a partner that I’ve given this box to, and now I don’t want that partner to connect anywhere else except to one device in the same original data center.
So I can even build a VPN that, what it does is a point-to-point topology. So when I meant there’s anything you could do, I meant it. Any topology that you can envision, right? It also has an implication on application performance. Because, if I want to build a full mesh topology, I may reserve that for unified communications, it’s going to get an utmost experience, right? But I may have other applications that work better in hub-and-spoke topology.
So it’s not just a security mechanism, it’s also a mechanism to have a better application experience. Now, another interesting case, which is exactly what I’m going to show, is how do I insert those services into – and why do I need to insert services? Like, what is it exactly that I’m trying to do? Service insertion sounds cool, or service chaining sounds cool. But what exactly am I trying to solve here?
So the way we look at this is establishing a regional secure perimeter. So I have my central data resources, and those could be on-prem data center, or could be a cloud data center. And then I have my fabric, right? So what I’m really trying to get to is I’m trying to establish this perimeter around my protected resources. So, should something happen in one or more of those sites, how do I make it stop, right?
I have a virus outbreak, or I have some malicious activity. How would I make it stop? Do I disconnect this site? Okay, then how am I going to clean it up, right? So, the thought behind this in here, is that I’m going to steer this traffic of interest into the services that are deployed regionally, and I can steer them to their closest location. So if I’m a West Coast, sort of infected branches, I can steer them to the West Coast regional facility.
If I’m East Coast, I can steer them to the East Coast regional facility. I can steer them to that, and in that facility, I’m going to hand them to the firewall. Firewall will inspect it, return the traffic back to me, I’ll continue executing. And I’ll continue delivering this to the data center. If the traffic was allowed. If the traffic was disallowed, I’ll never get it back. It basically dies on the firewall. Right? And that’s exactly what I’m going to show you.
Two sites. They can communicate. What I’m going to have is – two services are inserted into the fabric and are being advertised into the fabric. Attack starts happening, and I’m going to use TCP syn flood. Policy is applied. It’s chained through two services. But guess what? Firewall in this case is not acting as an IDS device. So TCP flood is happening, yet firewall does not have the policy to stop it. It could, because Palo Alto is a powerful device, so it could.
But in this case, I want to demonstrate that it doesn’t. So, it actually allows this traffic to go through and then it’s going to hit Snort device. And a Snort will block it. So think of that as a chain, and not just a service insertion. Again, VPN-aware. So, like we said, VPN is a very powerful tool. I can have this behavior in just one single VPN and I can have a different VPN that have no service insertion. And I have a third VPN – so it may have its own full mesh [unintelligible 0:12:22].
And I have a third VPN that has a hub-and-spoke topology. No services. So, everything is completely sort of isolated. It’s up to you how you want to carve behaviors in those VPNs. It’s a very powerful concept for enterprises that are trying to determine how their network is behaving. It’s not just a blanket behavior over my entire network – I insert the service, it’s everywhere – no, I insert a service in a specific VPN for a specific application. Extremely powerful and extremely granular.
So, I’m going to go back into the thing, into my desktop. I’m going to start this [angry man] in here. So, what it does, it basically starts sending a whole lot of [TCPC] packets, between the two sites, right? So now what I’m going to do – there’s a Palo Alto and …let me just show you what Palo Alto looks like. I’m just logging in to Palo Alto. The virtual Palo Alto appliance. So, it’s going to have basically a policy that says that site, going to a remote site, going to the data center, allow.
Because in this case, we want to demonstrate that the firewall is not going to be the one that is going to stop it. So, in the policies in here, I can see the policy that says Site One to data center – any service allowed. So, firewall is not going to stop it. Yet, IDS is basically – there’s really no configuration needed. Yes, you can have different signatures, but by default, Snort is a pretty powerful tool to just detect malicious activity with – and just accident, right?
So, think about a Snort that’s having sort of – TCPC is a pretty basic behavior, so Snort will have no issues without having any specific intelligence to detect it, right? So, what I’m going to do now, is I’m going to just show you one thing is that if I go into the device where my source is located, which is the Site One, I’m going to say show me the information that you know about the rest of your network, which is what’s advertised through the OMP.
And I’m going to say, show me how you connect into the data center, which is 182.168.4. And it’s going to tell me, is that 182.168.4 is actually available through an identifier of a data center vEdge. So, I know I’m going to the data center, right? So now I want to alter that behavior and I want to make sure that it doesn’t go to the data center anymore, it goes to the services. The question is, where are the services?
So if I go to the device which is, I designated it as the regional data center device in here, and I say can you show me an OMP services? It says yes, I have two services connected to me. I have net service one and net service two. Why do I have two services connected? Firewalls are [stateful] devices. When we send traffic through the firewall, we want to make sure that we are not breaking the flows when we send it to the firewall.
So it’s not just sending to the firewall, because the security teams are the ones who are actually configuring the firewalls, and the firewalls are very common – in fact it’s kind of mandatory – to have trust and un-trust zones. So we want to make sure that when we steer traffic to the service, which is aware of the state, we steer this in a stateful fashion. That’s why you see service one and service two being advertised.
So think about it: as I took a firewall, I connected it to the vEdge, it’s connected in the regional facility, it’s now advertising service one and service two – the vEdge is basically advertising to the rest of the network, here if you come service one or service two, I own it. Okay. Now, a different device in the network – and by the way, it’s also kind of interesting to note here is that note this originator, 10.10.10.25.
Keep that in your mind. This is going to be the identifier of where the firewall is. Because if you remember, the identifier before that was 10.10.10.14, which is the data center – this guy has 10.10.10.25 – so I’m going to show you, once the policy is pushed, I’m actually going to say that oh, now this advertisement is acted and it attracts the traffic instead of going to the data center, instead of going to 10.10.10.14, it’s actually going to go to 10.10.10.25.
So this is the firewall. And now, a different device, which is the regional data center two – so they could be completely locations, right? I can see it advertised as a service called IDS. So these are two completely different locations. They’re all hosts, their services, they’re advertising this. Think about this as also security as a service type of offering. I can offer a [farm] of services out of locations, and then consume that.
So that works for an enterprise customer for internal deployment. It also works for a service provider customer that wants to leverage this as security as a service offering. Advertise into the overlay, advertise into a VPN, where your customer is, and then that, build a policy that takes the traffic from the traffic of interest, and steers to that appliance. So it’s not just for enterprise consumption, but it’s also for a service provider to offer as a service.
So now, what I’m going to do is I’m going to go into the policy and activate the policy. Now I’ll go back into the – as the policy is being activated, I’ll go back into the – actually, oops. That was my host by the way. I’ll just go back into it. Okay, I was kind of late. It was already active. You see the TCP flood stopped. Because the policy was already activated. So see? No more TCP. So what happened? Who did what?
Okay. So if I go now into the vManage back, and I go into the device and I say site one, that’s where the attack has originated, and I go here and I say show me the OMP information…how do you get to 126.96.36.199? That used to show 10.10.10.14, which is the data center. Now, since I pushed the policy, it says no, no, no – you’re not going to the data center – you’re actually going to the regional data center, and I’m going to hand you over to that service. 10.10.10.25. That was the identifier of the regional facility. We steered the traffic from basically four sets. Not IP routing. We didn’t just change a destination IP address. The routing stayed the same.
But with [unintelligible 0:20:01] the traffic, and we plummeted through the tunnel that goes into the original data center and we gave it over to the firewall. So, what the firewall saw was the original packet with the original source and original destination. Now what did the firewall do?
Male Voice 5: Dave, real quick. The firewall has got to have a layer to adjacency to the [telebox], right?
Male Voice 1: The firewall needs to have an adjacency to the vEdge, yes.
Male Voice 5: Yeah. [Crosstalk] you’re just going to, going to send it as a direct connect.
Male Voice 1: Yes. If I go to the monitor – kind of sluggish – but I see this activity –
Male Voice 7: Just one point here. So you can also do [unintelligible 0:20:45] with the service if the service allows it. So if you can configure –
Male Voice 2: So, it doesn’t have to be directly connected?
Male Voice 7: It has to be – [crosstalk]
Male Voice 5: – point you’re trying to get to, it’s got [unintelligible 0:20:56] capability. You could build the [Unintelligible] that way. Sure, I get it.
Male Voice 1: So you can see in here that the firewall has received the source 10.10. – oh sorry, 182.168.1 – go into four, the action was allowed. So it didn’t do anything. So that was not the firewall that stopped it. So what actually stopped it was – I have to hop into…so what I’m going to do now is I’m going to log in to the console of the IDS device, and show you that it actually took an action to – here, see this counter?
Actually, the attack is still happening. It’s just, data center is not being bombarded by those TCP [unintelligible 0:21:51]. All of those are dying on the Snort. If I look at what it’s matching, it’s actually matching – so I was actually doing this SSH, so the port was always 22. So, it did detect an SSH scan and it was an attempt at brute force attack, so that’s basically what the IDS reports to you.
So, we took a traffic that was originated from the remote side, going to the data center, that was a malicious traffic. We pushed that traffic into the regional data center one, where it got subjected to a firewall policy. Firewall returned the traffic back to us, because it did not block it. There was no rule to block this traffic. It gave it back to us. We sent it to another regional data center, or we send it to an IDS – maybe you’ll get some luck with it. And IDS locked it.
Do I have to do two data centers? Of course, not. I can deploy this in the same data center. But that’s just to show you that it’s completely independent of how the topology is set up. And of course, as I’m hopping from data center one to data center two, from site to the data center, all of that is completely built on the same principles of traffic independence. So how I constructed my network – am I using MPLANs, am I using broadband, a mixture of the two – just broadband, just MPLANs – all of that is irrelevant.
What’s relevant is I’ve been able to achieve this security perimeter in a way that allows me to chain this through two of more services.
Male Voice 4: [unintelligible 0:23:25]
Male Voice 1: Okay. So let’s see – by the way, do you have questions? I know it’s a little bit, kind of loaded. So we’re not kidding when we said it’s [enter] first grade. [Laughter]. That is what the enterprises that we are talking to are asking us to do. We didn’t just build this because we woke up one day and we felt that this would be cool to do. Which it is cool to do, but that’s not the main thing. [Laughter]
Male Voice 8: So one challenge I can think of – I have some customers that are doing, they’re all across the U.S., but they don’t have regional data centers. They’re doing DMVPN today with like, Cisco, CWS – do you have any integration partners that do web security?
Male Voice 1: So, first I’ll answer the regional data center. So, what is a regional data center? It’s not some magical place that things just happen. For us, we treat every member of the fabric as an equal member of the fabric. So, when I mentioned regional data center, it doesn’t mean that it’s an actual data center. What it means, it’s a well-connected site that you decide – when I say well-connected, I mean it has enough bandwidth – because now, it’s going to become an [incast] for the traffic that goes to it.
You ask an administrator of that system, or somebody who designed the system decided that this site has enough connectivity to accommodate all this incast of the traffic, and you are going to provision services in there. It doesn’t mean that you’re going to have to go and redesign your network and start building regional data centers. You can just designate facilities that already exist in your network in our SD-WAN facilities, because of course you need an intelligence of an SD-WAN fabric to do this. So, you have a site that has already migrated to an SD-WAN site, and you decide it has good connectivity – either broadband, MPLS, both or whatever – has enough bandwidth to accommodate the incast of traffic and that’s where I would put a firewall.
And you could say, I’m going to put one in Seattle, one in San Francisco. For redundancy. Then one in Austin, one in Chicago, one in New York, one in –
Male Voice 8: That’s fine for some organizations, but I mean, okay, say you’re a Seattle-based organization. You build your first data center in Seattle, you build your second one, sure, San Jose, wherever. Well, you’ve got offices in Georgia. You’re not going to tunnel your traffic back to San Jose or Seattle –
Male Voice 1: That is true.
Male Voice 8: So that’s why I was asking if you had partners that did web security that have regional data centers.
Male Voice 1: Yes. So, we’re probably going to run out of time for that, but –
Male Voice 8: We can talk about that off [unintelligible 0:26:09]
Male Voice 1: A Zscaler, right?
Male Voice 4: [Crosstalk] right? One is certainly Zscaler so – actually, one of the demo we should show [Crosstalk]
Male Voice 1: Yeah. I can show – [Crosstalk]
Male Voice 4: So that’s one. The second one is we do have a whole lot of value-added security partners who actually host this as a service and can spin up an instance of a vEdge right in front and insert back into the offering. So both of those options are…
Male Voice 1: So, Zscaler is basically a connectivity to the cloud-based service. So, if what you’re looking for is a cloud-based security – so what we’ve shown you basically, it’s an on-prem security, or security as a service offering, right? But it does rely on actual boxes. Maybe virtual boxes, but it’s a piece of security infrastructure, right? So, if you’re subscribed to that, then service chaining is your friend.
If you are in the locations, just like you say, which is remote locations, and you don’t feel comfortable backhauling this to wherever the security service is, then yes, you can subscribe to cloud security, such as Zscaler, that will be your friend.
And again, it’s on per-VPN basis. So, you can have VPN one that is doing this, and you can have VPN two that is taking it to the Zscaler. And I can allow VPN one and VPN two to communicate. So this is, if I’m creating VPNs, but I’m not really looking for segmentation, I’m just looking for different behaviors, I can do that too. So it’s a very powerful sort of way of doing that. So the way that I can show you is basically – if I go here and I just go, let’s say, I’m going to start this ping in the background – I’m going to come back to it shortly.
So, something is running, pinging the internet. So what I’m going to do, I’m going to open the browser, go to two websites – Wikipedia works. BitTorrent. A little bit slower, but it works, right? So, no security, right? Basically, there’s nothing in between that blocks my URLs or anything like that. So, what we have done is – you like to do Zscaler integration, of course you can’t do Zscaler, they provision an account for you, they give you the information about how to build your tunnels, what is your endpoint.
If you ever provision Zscaler service with them, then you would know. What you do on the vManage side is that you basically create a piece of a template in here which is basically says this is your source interface for your tunnel, this is your destination interface for where you’re going, this is the Zscaler POP – all of that information you get from Zscaler – this is the IP address you should put on your tunnel, all of that the information you get from the Zscaler.
And, of course, you can apply other things such as ACLs on it and [policers] on it as it goes to the Zscaler, right? Zscaler offers that functionality to, but if you don’t want to subscribe to that – all you want is just URL filtering – you can just do a log in to the Zscaler portal, just enable URL filtering, and do your ingress policing, ingress ACLs or egress ACLs and egress policing by yourself, right?
So, what this interface is, is this interface was shut down, right? So when I un-shut it –
Male Voice 4: And this is a template, which means if I have hundreds of devices, and all hanging off of the same [unintelligible 0:30:01] or to interface, it’s a single template change and you have a hundred sites now that can just contact Zscaler.
Male Voice 1: Right. Yes.
Male Voice 4: In one chart.
Male Voice 1: Right. And the important thing about Zscaler is…why are you doing Zscaler? Why are you doing cloud security? One element is that you don’t want firewalls. But there’s another element, which is exactly you brought that too: I don’t want any backhaul. So if I contact Zscaler, yet I need to backhaul somewhere to get to Zscaler, that’s kind of counter-productive to the existence of the Zscaler, right? Zscaler’s value proposition is yes, it’s a cloud security, but it’s the most effective, the fastest way to get there.
So I don’t want to tunnel somewhere and then get to Zscaler from there, because that’s pointless. By doing templates-based configuration, I’m now able to, from every single site, get out. So there’s no administrative burden, because it’s all templatized, yes I get the fast paths to get there. Because I’m not backhauling it through any other device somewhere in the network to get to the Zscaler.
Because that would be counter what Zscaler would want you to do. So it’s been provisioned. Basically, there’s a GRE tunnel that was activated. And if I go back into this guy, and I just reload, I’m going to get this nice pop-up that’s basically says you violated the policy – that means I’m going through the Zscaler. And if I go to Wikipedia, it still works, right? But here’s the fun part. Remember the ping that I was running?
See this. Zero packet loss. So what I’ve done is I’ve introduced – if I were tunneling all my traffic to the data center, and letting it go from the data center, because data center is the master, the center of the universe. That’s where my firewalls are, and URL filtering are, things like that. Now I want to move to Zscaler. I call the Zscaler, I establish the connection to them. I provision the connection on the vEdge infrastructure with templates.
And now I steer the traffic to the Zscaler. And I got zero packet loss on that service rollout. That’s how you create [enter first grade] solutions. [Director’s comments] All right. Let me talk about another –
Male Voice 9: May I ask a question?
Male Voice 1: Yeah. Absolutely.
Male Voice 9: I forget what the controller device is called –
Male Voice 1: VSmart. Because it’s smart.
Male Voice 9: Is there any requirement for, like, persistent connectivity for branch offices to connect to that? And if so, what happens when they lose connectivity?
Male Voice 1: Yes, a very good question.
Male Voice 4: While you’re pulling that, maybe I can answer.
Male Voice 1: Yes, please.
Male Voice 4: Okay. So short answer is every Edge device has an active/active connection to say a pair of controllers for redundancy. That’s what you use to propagate network-wide information. Network-wide information. Now for some reason, if your connection to a VSmart goes down, the device hangs on to the last-known good state, so your data plane connectivity still continues to function. Obviously, you’ll not be able to make any management changes, or you won’t be able to propagate network-wide routing information, but your data plane continues to function.
Male Voice 9: I don’t lose the data plane, though.
Male Voice 2: And you don’t lose the data plane. And you can configure [unintelligible 0:33:37] up for two hours, two days – you let the system know that if I, for some reason I lose all connectivity to all the controllers – I want to hang on to the last-known good state for a certain amount of time.