Making Private Network Connections Over Cellular IoT Networks (with Felix Hsieh)

This was originally an internal conversation that I had with Felix Hsieh, a product manager and customer reliability engineer at Soracom. The discussion was during a monthly AMA (ask-me-anything) virtual office hours call. I came to the call with the desire to learn more about the ways in which private networking has been made more accessible than ever before for cellular data connections and entire networks of deployed IoT devices. 

In this discussion, Felix shares how private networking over cellular has historically been challenging, with lots of red tape, delays, contracts, and technical hurdles. We discuss the evolution of support for virtual private clouds, the role regulations can play on project requirements, and review scenarios of how different companies across a variety of industries implemented their private networks over cellular connections. 

Please keep in mind that this was an internal conversation, so Felix will often talk about how Soracom engineers addressed some of the traditional barriers that setting up VPNs and VPCs that most cellular carriers actually still struggle with today, so it's not a sales pitch, but I felt that it's worth calling out upfront. 

One other note to call out is three acronyms are used within this episode that are incredibly similar to one another – VPN, VPC, and VPG:

• VPN, which we all should know as a virtual private network
• VPC, which is a virtual private cloud; unique to AWS and its secure web hosting. 
• VPG, which is a virtual private gateway; a feature that Soracom leverages within its connectivity platform to bridge private networks from a customer device in the field to a customer's virtual private cloud instance or corporate VPN network. Seamlessly without touching the public internet. 

Transcript

Ryan: 0:00

Welcome to conversations in connectivity. I'm Ryan Carlson, your host. This is a podcast for IOT professionals on the IOT curious, responsible for growing executing, or educating others about connectivity and the myriad of moving parts associated with connectivity operations. This was originally an internal conversation that I had with Felix Shea.

0:18

A product manager and customer reliability engineer at Soracom during a monthly, ask me anything virtual office hours call. I came to the call with the desire to learn more about the ways in which private networking has been made more accessible than ever before for cellular data connections and entire networks of deployed IOT devices. In this discussion, Felix shares how private networking over cellular has historically been challenging with lots of red tape delays, contracts, technical hurdles, and we discuss the evolution of support for virtual private clouds, the role regulations can play on project requirements and review scenarios of how different companies across a variety of industries implemented their private networks over cellular connections. Please keep in mind that this was an internal conversation so felix will often talk about how Soracom engineers addressed some of the traditional barriers that setting up vpns and vpcs that most cellular carriers actually still struggle with today so it's not a sales pitch but i felt that it's worth calling out upfront. One other note to call out is there are three acronyms that are used within this episode that are well incredibly similar to one another. There's VPN virtual, private networking there's VPC, which is virtual private cloud, which is unique to Amazon web services and their ability to have secure web hosting. Then there's VPG, which is virtual private gateway a feature that Soracom leverages within their connectivity platform to marry. Virtual private networks across virtual private clouds. Seamlessly without touching the public internet. There you go. VPN and VPC. VPG. and now onto the interview.

Ryan: 2:09

So big conversation lately has been the virtual, private networking world. just in general, I think the conversation is it's not just about our virtual private gateway service It's not even limited to Amazon's Virtual Private Cloud. It's building a secure IoT infrastructure so from an operational perspective, I'm trying to wrap my head around all of the various moving parts on who's involved and who's invested in that happening. I'd be curious on the other side of the pond, whether there's, examples that you have, where organizations are focused on operational security, or even infrastructure security, how are they viewing what SORACOM services provide. if we're right. I, I wanna elevate the conversation beyond Sims. And what is it that we're doing from a platform perspective to really address security concerns or anticipate security concerns?

Felix: 3:10

yeah. that's, that's a very good open-ended question. it, it's depends on the use case. there are a lot of different perspectives that are, relevant and for each type of industry that you're working in or whatever type of use case that you're looking to build, the types of, things that you're looking at, can actually be really quite different. I'm thinking we have a couple of customers, where. the particular area that they work in, and actually it's coincidental for both of these customers that they are, working in the energy sector, but they have, government mandated regulations. where they, the, the type of insight that there, or the type of, infrastructure that they're building for monitoring and managing energy systems or energy production, is, understandably a concern for the government itself. So they have government mandated regulations for. securing the, that infrastructure, in some sense, it's like a matter of national security. And so they have regulations for private networking, uh, you know, using private networks in order to manage that infrastructure rather than using public internet infrastructure. so you have, in some se, cases, government regulations that are. pushing or requiring the use of, or development of private networking for infrastructure. in other cases it may be some kind of a separate, independent, regulatory body or, a working group. So for, example, in the cases of, payment transactions. So we have a number of customers who do a point of sale credit card transactions, that type of stuff. Then you have the PCI organization that, has their own compliance, regulations. these are the regulations that you need to be aware of and basically you can take every, any particular, vertical or industry. For example, if you use, if you end up using part of our technology such as PPG or whatever, it is that helps you create a virtual private network. Then, there are certain things about regulations or, compliance where you don't have to worry about, Validating your implementation, like spending time to actually validate your implementation because the architecture is actually, is already built from the ground up in a secure way where you don't have to perform sort of an audit or something like that. Yeah.

Ryan: 5:42

One thing I did want to bring up was thinking about the product development and life cycle. specifically around dependencies. so thinking through our platform services and a potential project, where is the point of no return? Okay. So when is it too late to implement? Like we know our device certificate management features are something you would want to implement fairly early, right? You could go down a rabbit hole. So in regards to things like virtual private networks or private cellular networks, how soon does that decision often need to be made from a pilot or prototype or proof of concept

Felix: 6:22

into that first pilot? that's actually a really interesting question traditionally. you would need to make that decision quite early in, in the development of an IOT architecture. mainly because. Before. it's a little bit like a chicken and egg. Like how do you get your engineers to develop something? If you haven't decided whether you're gonna be using AWS or Azure or Google cloud or whatever. And so you have to make a fairly early decision. our company, we're going to go all in on AWS or we're gonna go all in on Azure, whatever it might be. and if you maybe happen to know, ahead of time that at some point you're going to need, you're likely going to need a virtual private network for your solution. Then you have to pay attention to, the types of services. Like one of the, there are a lot of different, components on these sort of major cloud providers that some of these components do support private networking, and some of them don't. as an interesting example, AWS I O T is one of these components that is, particularly good, like parti particularly well suited towards devices that are constantly connected and just, relaying messages or relaying data back and forth. and so you can have sort of a data management of IOT devices at scale, uh, but AWS IoT did not have, private networking support for quite a while, at least until I think about a year and a half ago or so. and so if you were a developer you had to actually go in and. Figure out all these details like, oh yeah. I see all these great blog posts about how AWS I O T is a great solution for managing data of, tens or hundreds of thousands of devices. And we're using AWS and we'd like to, use AWS. I OT as part of our solution you need to know ahead of time, a couple years ago, you would need to know ahead of time that heys, I IOT doesn't actually support, private networking. So that, that's kind of, you know, before you get too far in building your architecture, you have to actually plan everything out, in advance. so

Ryan: 8:30

is that still the case though? Does

Felix: 8:32

it just, cause I know they've got,

Ryan: 8:34

AWS, IOT's got end device end points. Are they just not VPN

Felix: 8:39

compatible? I should say, is the idea that SORACOM is providing the infrastructure that is connecting two separate endpoints. And the two endpoints are both up to the customer to decide one endpoint is of course the device itself. And then the other, endpoint is whether you want to use your own AWS environment or Azure or GCP, or even your own data center or whatever. and, if you're using, a cloud provider, cloud providers like AWS and Google and Azure, they are constantly, Updating their infrastructure updating their services. just like with the AWS I O T example, eventually, AWS did come around and add support for AWS I T for, private endpoint. So you can actually put AWS, IoT inside of your own private cloud inside AWS. And then you can actually, accesss I T from within a private network. So there, these things are always changing, always improving. unfortunately I'm sure if you, I'm not sure if you caught wind, but, Google is actually end of life in GCP, uh, Google IOT service. So that also is a issue that developers need to pay attention to. but in any case, the traditional way is that you would have to know all of these components well in advance and then plan for everything well in advance before you can even start developing before you can even start building it. And so if you have some kind of a regulation or some kind of compliance requirement that says, Hey, your architecture needs to be sending the data over a private network, you would have doing that sort of at the very early sort of blueprinting phase. Now, where SORACOM comes in is that our infrastructure, our virtual private network, like our VPG and then our services such as canal and door and direct, these can actually technically be added after the fact, so they can actually be added later on, on top of an existing architecture, without too much reconfiguration.

Ryan: 10:47

And when you say canal Door and Direct referring to the Soracom platform features or cloud connectors That allow a user to even retroactively go back in time and add a virtual private network from their device to cloud implementation.

Felix: 11:03

Yeah but, that, that doesn't mean that the blueprinting is therefore, no longer relevant you can, you would still have to know an in advance, if, for example, AWS IOT supports private networking or not. So let me just give two quick examples. Let's look at the example of AWS IoT two years ago, where it did not support private networking. So we actually had a customer that went through this exact process. They are using, AWS, I OT as their backend, and they have a whole bunch of devices that are deployed out in the field. And they wanted to, eventually migrate over to, using a private network solution. And before coming to SORACOM, they had an existing solution together with AWS, I O T but they were just going straight over the public internet. So their devices were connecting to a public internet gateway and they had their AWS, I T a. backend configured with its public internet endpoint and the devices. We're just connecting to those endpoints over public internet, then come along. And what they can do is they can actually, as they're SW swapping their Sims for SORACOM Sims, they can enable a V PPG that can create a V G and then say that these Sims belong to this V G and then from that V G they can use SORACOM canal to create a private network connection over to, their AWS environment. And so suddenly all of the data initially is going from, the device to SORACOM over to a private network inside SORACOM, and then over the private network connection from SORACOM over to. a customer AWS environment. Now the problem at that 0.2 years ago was that, the AWS IOT backend itself still only had an input from, the public internet so in any case, they, the private network connection was set up, but then once it got over to, HS, AWS environment, it had to take a quick exit back out to the public internet to get back into AWS IOT. Now fast forward, about a year ago when AWS introduced, VPC endpoints for AWS, I OT. Now you can go directly to, AWS, I O T from directly inside. So our private network can be added or removed after the fact. And that is actually quite a nice thing to be able to do. But at the end of the day, it still doesn't get around the fact that you would need to know at some point in advance. Ideally you would like to know at some point in advance, if you are. Architecture is going to require a private network in the first place. Because again, if it's something like AWS, I T if you need to know, if you need to have a virtual private network, you would've needed to know well in advance that at least two years ago, AWS, I T did not support, connections coming in through a private network.

Ryan: 14:07

Yeah. So let's let's say for a moment, we have someone who they've been just encrypting data at the device level, prior to sending it over cellular So I already know that at a device level, if I wanted to setup a virtual private network through a traditional wireless carrier, if you were to go with a, like at and T and they set up a VPN, you'd need to issue credentials down to the device level. That's right. So there's some device level shenanigans that needs to happen. Yes. You have to touch the device and do something with it. Yeah. whereas if you're switching to SORACOM mm-hmm and you're putting SORACOM Sims at the device, that is in essence the same level of, it's actually less effort than having to go through the credentialing at the device, just put in the SIM, and now it's actually putting all of the secure. VPN stuff in through our

Felix: 14:57

network, That's right. Yeah. So what we are doing, which is different than pretty much every other MNO or MVNO out there, there's probably only one or two other MVNOs out there that do a similar approach as us. Okay. So we are in very limited company. We add a layer of abstraction on top of just, the existing cellular, infrastructure, the cell, cellular, technology. Yeah. which means that when you. When you want to set up, let's say private networking, with that layer of abstraction. You have the ability of controlling that by just logging into console.SORACOM.io. Rather than having to go all the way to the, to each individual device. As long as your devices already have a SORACOM SIM inserted, then after the fact, you can just log into the user console and then configure how you want that connectivity to be managed or to be handled, whether it should go over the public internet, or go over a private network. that's something that you can change after the fact without having to, metal with each individual device. Oh, so yeah. So the.

Ryan: 16:12

The SORACOM put, having, having a SORACOM SIM is like having a little bit of a time machine for technical debt or like decision it's like we can put off making some of these decisions. Yeah. Because it's not irreversible.

Felix: 16:25

correct. Yeah. Oh, okay.

Ryan: 16:27

I'm trying to think of it from a, CFOs perspective or, chief operating officer, someone who actually sees all of the truck roles or the things that would need to be required to make changes to infrastructure later. so it's, you're not married to it, but if you ever wanted to go down that road, It saves

Felix: 16:45

a lot of that hassle. Absolutely. Absolutely. And and yeah, definitely as for startup, like if you wanna say from the finance perspective, one easy kind of call to action is managing your runway length. if you have, if it's, if you're playing in an industry where you can get away at a small scale without having to commit to, Fairly expensive contracts for private networking. Let's say, if you wanted to go with at and T as in your example, you said at, and T I'm building something, it, I have these regulations or compliance issues and I need a private network, number one at, and T's gonna be like, okay, what's your commitment level? And you're like, we're a startup and we're pitching to investors right now. We don't really have a lot of runway length right now. So that's your first issue. And then of course, on top of that, once you do get to a point where at and T is okay, yeah, we're willing to, for that commitment level, we're willing to put together a VPN for you. Then, of course on top of that, you have the reconfigure, the device reconfiguration issue as, that, that will come up later as well. and so for us, SORACOM was like, yeah, we don't care. whether you want it for one SIM or you want it for a hundred thousand Sims, like just create it whenever you want and then reconfigure it whenever you want and you don't have to touch the devices. So definitely, you can look at it as a value add from that perspective. the flexibility is very, powerful.

Ryan: 18:07

I see the scenario of why someone would be approaching SORACOM is either they aren't being given the time of day by by a large carrier or there's the big commits or, we're just, we're easier to deal with at the R and D level. But someone who already has deployed devices, I don't see them coming from an MVNO. They're probably coming from a, an at and T or T-Mobile where you know, like, Hey, the service you get is a service you're gonna get. And the technology is the technology and no matter what happens, you still don't get the keys to the console. You still have to go with hat and hand to your rep and go, please, sir, can I have a VPN?

Felix: 18:49

Right. Well, another side of this is definitely for customers that are already at scale, or they have a particular application or use case, which is already very. Stable, it's in an industry where the regulations are very well established. Their existing application is compliant with that. It's not subject to, ebbs and flows or like the changing regulations that would occur. these are things that are very stable and they're, they don't change very much over, a long period of time. That's definitely where the flexibility doesn't have as much of a value, a perception of value. if I've already got 10 million devices out there, let's say for example, shipping container tracking is one of these, use cases, right? Where, all of these different, companies, they already have tens of millions of trackers on these containers. for that 10 million SIM, commitment. They're more than happy to dedicate a team, a small team of, technical account managers and support staff in whatever, to build out and maintain the VPN for that customer. And so you can have those types of customers look at SORACOM, be like, why would I wanna do that by myself when I can have a team dedicated to do that for me, And of course, if you have, an application where the regulations aren't changing very often, and you've already built out a solution that incorporates a VPN that is provided by Vodafone or at and T or whatever it might be. Yeah. then it's like, the flexibility of being able to turn it on and turn it off at a win. like for me, I just turn it on and forget about it. And not only that, I don't even have to turn it on my account manager, turns it on and handles it for me. So definitely not as valuable as a kind of selling point for large customers that are at scale. And especially if their applications are already very stable. Now, what I would say is that you can have customers that are at scale, but because of incoming regulations, that might be, that might be imposing more, the higher, requirements for data protection. they might be looking at that and saying, okay, this is the time. Like for example, we have a hundred thousand devices that are on two G 3g only right now. and in, in addition to, the two G 3g sunset, we have new updated regulations that are coming down and that are saying that, encryption and public internet is not good enough. We have to do encryption and a private network And if we're gonna have to be ripping and replacing, two G 3g modules over to LTE modules, because of the two G 3g sunset, we might as well just design a new device and start end of lifeing our old legacy devices. And so you can have customers that are already at scale, but because of the changing times, that becomes their sort of impetus or their driver for, implementing or updating their architecture. And in that process of implementing or updating a newer architecture, they might perceive the flexibility of being able to add or remove VPN connectivity using our layer of abstraction as another, as a value add. The beauty of our abstraction of this ability to control your infrastructure from the user console and not having to go all the way to the end device and reconfiguring the device is that this means that it's hardware agnostic. So it doesn't matter to us if your device supports VPN or doesn't support VPN, You're configuring that from the user console anyway, so to give an example, Teltonica is one of these, OEM partners and we have their gateway on our marketplace. the Teltonica router itself does quote, unquote support VPN capabilities. But the endpoint, if you were to go to the Teltonica router, log into the router and then configure the VPN settings, then the two endpoints that you would be creating is one, the endpoint is on the Teltonica router. And then two, the endpoint is on your, let's say your, a AWS, private network. And that's where the end to end, tunneling would occur. The private network would be starting at the Tica and ending at AWS for us. what you would be doing is you would actually be creating a different tunnel and that the tunnel would be between our packet gateway inside the cellular network, and then your AWS environment and whatever is going through, whether it's a tele router or an Arduino or a raspberry pie, an Onyx, whatever it is, then all of that data would go to the packet gateway inside SORACOM, which is your VPG. And then that is the private network endpoint, input endpoint to your, AWS VPC. So by having that layer of abstraction, it doesn't matter to us if your devices necessarily support, VPN or not. And that's one of the, if we double back now and look at, uh, ATT as an example, that is why, if you were to set up, at and T for private networking, you might have to do some device reconfiguration, because you might not be getting that same endpoint, with a VPN from at and T. but the other side of this though, and this is where the OEM side messaging might be a little bit tricky. there are definitely, gateway manufacturers that produce these, gateway devices, they're competing with each other. And in order to, differentiate their products, of course, they want to say, our, router support, VPN connectivity and such. And then if we come in and we say that doesn't matter, we have one layer of abstraction above that where we just intrinsically. build a private network for all of your devices, regardless of whether your device supports VPN or not. then the, that kind of removes that selling point. So, uh, a SORACOM customer that knows that they want to use the SORACOM VPG to build out a private network that then says, okay, which router should I use there? Isn't that meaningful differentiation between, anyone else because you don't need to use the, routers VPN capabilities anymore.

Ryan: 24:53

This is gonna get, this is a hypothetical storytelling land let's pretend that we are an engineer where we're building out this, proof of concept. And we're going to Digi key and, they happen to fall on SORACOM so they're early in their development process as an engineer. What are the types of

Felix: 25:12

things that you would

Ryan: 25:15

wanna see? other than, Hey, there's a SIM and oh, you can get sell your connectivity. right now you can send your data directly to your IOT cloud. So if we're thinking dependencies and right now I'm just, I just have to build a proof of concept and a prototype. What are your recommendations?

Felix: 25:31

So we have some copy and some, content related to how the SORACOM platform helps you connect to specific types of, cloud providers. my takeaway with that is if I were in an engineer perspective, I would be looking that and I would be thinking, okay, that is an application level service, meaning that if my IOT use case happens to use that application, that might be a valuable. information for me to see and to learn about and understand that SORACOM has the, the services that help me build that particular application more efficiently and more quickly. Yep. but that's only application if I happen to be working in an IOT use case where I don't want that application, or I don't need that application that is completely irrelevant to me. And I'm missing the point that there's this entire other world of the infrastructure, not the application, but the underlying infrastructure, that SORACOM is still also providing to me. For an engineering persona you're writing code that can do that programmatically. And I think what we. Might consider doing is saying programmable cellular infrastructure or something, a programmable infrastructure, because if you're at the application level, you're saying like, oh, we can connect to AWS. I O T engineer, I'm not using AWS. I T so that's irrelevant to me, or we can, I'm not using Google cloud, so that's irrelevant to me. But if you say programmable infrastructure, and then you say, well, what do you get with programmable infrastructure? Well, one of these things is for example, programmatically enabling or disabling a private connection. Suddenly that is something that reaches a lot deeper. A lot of different use cases. because as an engineer, if you're building an IOT solution, you look at this as like a bespoke, challenge where you have to build every individual component by yourself. And that's the whole point of the blueprinting phase where if you built something and then you need to go back and change a bunch of components in the middle that's a nightmare for your engineers. Yeah. So you wanna know in advance the right way to do it. if you can do it in such a way that you can programmatically automatically switch, for example, private networking on or off, depending on whatever situation or scenario that you encounter, then your engineers will look at that and be like, why would we manually build something when we can just use, some kind of API command that SORACOM has to just turn it on and turn it off? it's a bit of a reduction of simplification, but I think that, the next kind of evolution that we would want to push forward in front of customers is the fact that we are exposing the infrastructure and I say exposing, not in terms of security exposure, but we are giving control of the infrastructure over to the customer. And that's something that most MNOs and most MVNOs, simply do not have. And again, if you go to an M and O Vodafone or at and T and you have a high number of Sims, you don't really care about having that infrastructure exposed to you because your account manager will do it for you internally. But, or for customers that are growing, and that can't access that level of service, because of their commitment numbers, or they are in a use case where their requirements are changing rapidly. Having direct access. could you just imagine, for example, PCI compliance, use case again, and maybe there's a new regulation that says, okay, encryption over public internet is no longer, good enough. We need to have encryption and private networking. And if you're currently deployed on, let's say Verizon, for example, and you say, okay, there's this new regulation coming in. I need to retrofit all of my devices and we need to get, private networking set up. Verizon's gonna be like, okay. Yeah. Give us two weeks to get back to you. And then we'll forward your request over to the engineering team. And then from there another two weeks before the engineering team, actually gets around to giving you a call and then figuring out what your requirements are. So you're talking about like a month long process or longer just to get, a private network request, with SORACOM. Yeah. Just log into the console and turn it on. whenever you need it. So thank you for everything, Felix. Thanks. Absolutely. I was a great conversation,

Ryan: 29:51

yeah,

Felix: 29:52

bye-bye all right. Talk to you later, Ryan.