Posted on January 29, 2019 by Jon-Eric Eliker
Outbound Internet access ("egress") for an Oracle Cloud Infrastructure ("OCI") Compute instance is easily achieved by assigning a public IP address to the instance and establishing a route in the subnet's route table that uses an Internet Gateway. However, frequently our Compute instances have no requirement for (nor do we desire to allow) direct inbound connections ("ingress") via its public IP address. This article explores the four additional ways you can provide Internet egress to a Compute instance in addition to public IP address/Internet Gateway method already mentioned. You’ll find these five methods described below:
Assign a public IP address
As stated, this method is entirely the simplest method to granting Internet access. In fact, if you do nothing exceptional when launching a Compute instance in a public Subnet, you will be given an ephemeral public IP address by default. Note that you may override this behavior on the Create Instance UI from the Advanced Options > Networking section by unchecking this assignment of a public IP address.
Add public IP address to existing Compute instance
What if you've already launched the instance in a public subnet but did not assign a public IP address? No worries as you can add the public address without any downtime for the Compute instance. Do this by first locating and selecting the Compute instance in the OCI Service Console. Then, click "Attached VNICs" link under the "Resources" menu.
From here, click to modify the VNIC to which you will assign the public IP address. Note you likely will be selecting the "Primary VNIC" if your instance has more than one VNIC attached.
From the VNIC page, choose "Edit" from the menu to the right of the VNIC information.
Here then you can select an "Ephemeral" or "Reserved" public IP address for this instance.
Click "Update" to save the public IP address assignment.
Default route to Internet Gateway
The second step required is to ensure you have a default route defined sending traffic to the Internet. Note that "default route" implies the 0.0.0.0/0 CIDR destination (i.e., anywhere else) that is used if no other explicit routes are provided in the route table.
Caveats to the Public IP address/Internet Gateway method
Though this method of providing Internet access is very simple to implement, this "simple" choice comes with a few caveats for you to consider:
Perhaps this is obvious, but it is important you fully understand this requirement. You may have wisely defined private subnets to contain your Compute instances that do not need inbound connections from the Internet. Therefore, you do not have the choice to use this method to grant outbound Internet access to instances in the private subnet.
Foremost, Security Lists must be carefully configured to prevent unintentional inbound traffic across the public IP addresses you assign. Next, note that the Security Lists for the subnet containing your Compute instance must allow egress either to CIDR addresses of specific destinations (e.g., 220.127.116.11/32 or 18.104.22.168/24) or to a "default" destination (e.g., 0.0.0.0/0). Remember to either specify "stateful" rules (i.e., do not check "stateless") or ensure you have created the corresponding ingress rule allowing responses from your outbound connections.
In early October 2018, Oracle introduced the NAT Gateway as a companion to the Internet Gateway. Using the NAT Gateway as a target for your default route, you can provide Internet access to members of a subnet without having to assign each instance its own public IP address. Using the NAT Gateway, each Internet connection appears to come from a single public IP address shared by all members of the subnet that uses that NAT Gateway instance.
The NAT Gateway then handles the response traffic and ensures the responses are delivered to the Compute instance that initiated the outbound connection. This was a great addition, and welcome alternative to the more complex NAT model described below under "Use a NAT Compute instance."
Implementing a NAT Gateway is as easy as you might expect. Like with the Internet Gateway, you configure your "default route" for the subnet to use a NAT Gateway that you have added to your VCN.
Considerations when deciding to use a NAT Gateway
Keep the following points in mind should you decide you use a NAT Gateway for Internet access.
Unlike the Internet Gateway that applies to only public subnets, you may use a NAT Gateway on both public and private subnets. However, realize you may have only one default gateway (i.e., destination 0.0.0.0/0) in the route table for a subnet. Therefore, you cannot have some instances connecting outbound using the Internet Gateway (via their public IP addresses) and others using the NAT Gateway.
The title says it all: every outbound connection from the subnet served by a particular NAT Gateway will appear to originate from the same public IP address. This is by design with the current NAT Gateway implementation and cannot be changed.
The public IP address of the NAT Gateway cannot be changed or selected. The public address is automatically assigned when the NAT Gateway is created from a pool of addresses controlled by Oracle.
Inbound traffic to Compute public IP addresses is effectively "blocked"
Inbound connections to Compute instances that have public IP addresses require an Internet Gateway. Therefore, if you are using a NAT Gateway to serve outbound connections, you cannot also have an Internet Gateway on the same subnet with the intent of serving inbound connections only: only one or the other may fulfill the default route for the subnet. You may, however, provide inbound connections through other means such as load balancers but that configuration is beyond the scope of this article.
As an aside on that last bullet point: you could conceivably allow inbound connections to a Compute's public IP address from certain sources by establishing an Internet Gateway route serving only the CIDR address of those sources. In this use case, both a NAT Gateway and an Internet Gateway are in the route table but address different destinations.
Before the NAT Gateway was introduced, Internet access with NAT required deploying a Compute instance configured for and dedicated to this purpose. Oracle provided a white paper that described this process including a nice write-up on security practices to consider with such a configuration.
Beyond the configuration described in that white paper, some customers may have opted for a more robust "NAT instance" by using a third-party virtual appliance deployed in the OCI environment. In either case (simple NAT instance or virtual appliance) the process is generally equivalent. Therefore, we will use the terminology "NAT Instance" to imply either a simple instance as described in the white paper or a virtual appliance.
Here are the basic steps for this configuration:
Using a Compute instance as a route gateway
OCI route tables have a specific route target for this configuration: Private IP (vs. Internet Gateway, NAT Gateway, or others). As with the other target gateway types we've examined above, specify the default route CIDR 0.0.0.0/0 then specify the NAT instance which will handle the routing and NAT services. You may specify the NAT instance by either its private IP address or OCID.
Disabling source/destination checking on the VNIC
The VNIC of the NAT instance must be set to disable source/destination checking. By default, the VNIC will drop any packets that it receives that are not bound for that VNIC specifically. When used for NAT purpose, this check must be disabled so that the VNIC will, instead, forward these packets on to the instance(s) for which NAT services are being provided.
Disable this source/destination check by first accessing locating the VNIC from the Compute instance as described above ("Add public IP address to existing Compute instance"). Instead of clicking to select the Primary VNIC, instead, use the menu to the right to select "Edit."
Ensure the box "Skip Source/Destination Check" is checked then click "Update VNIC" to save the setting.
You may be surprised to find this method works properly (reliably) with a single VNIC. Though you may choose a more complex configuration by assigning the NAT instance a VNIC in each of the "internal" (NAT-protected) and "external" (Internet Gateway-routed) subnets, you should find this single-VNIC approach sufficient for most use cases. This is explained to some extent in the NAT instance white paper noted above.
Points to consider with a NAT instance
Here are a few characteristics of using a NAT instance that may affect your decision to select this path for Internet access.
While the Internet Gateway and NAT Gateway are implicitly "fault tolerant" (in that they are managed deep within the OCI infrastructure itself), a NAT instance that you configure and deploy is not. You bear responsibility for defining and deploying redundant NAT instances as appropriate for your infrastructure configuration.
The amount of bandwidth allocated to a Compute instance is a factor of what Compute shape is used when defining that instance. Though functionally, a one-CPU shape (e.g., VM.Standard.E2.1 for EPYC-class CPU and 8GB RAM) may be more than enough, the bandwidth available to that shape may not be appropriate for the amount of traffic you expect to pass through the VNIC of that shape. For example, the VM.Standard.E2.1 shape provides only a maximum bandwidth of 700 Mbps (megabits per second). Fortunately, you can increase bandwidth by choosing a larger shape—up to 24.6 Gbps for VM.Standard2.24!
This should come as no surprise since each Compute instance has a per-hour cost for OCPU usage regardless of its use in the environment. That cost grows even higher if your bandwidth needs necessitate a larger Compute shape. Also, there will be a minor Block Storage cost for the boot volume associated with the NAT instance you deploy.
See this section in the Oracle Cloud documentation that lists the maximum bandwidth available for each Compute shape:
"Compute Shapes" (oracle.com)
The Internet access methods described above all presume a subnet-wide configuration in which all instances have similar access needs. However, in some circumstances, you may wish to provide Internet access to only some (not all) of the Compute instances in a subnet. You can achieve this by assigning a secondary VNIC to one or more Compute instances and placing these secondary VNICs in a subnet that has an established means for Internet access (via any of the methods described above).
In the example architecture above you see that the Private Subnet has no defined routes or gateways (either NAT Gateway or Internet Gateway). However, because of the "connection" established with VNIC2 in Public Subnet, the host for that VNIC—called "Private Host" in this example—has a route to the Internet through the route configuration for Public Subnet.
Consider the following for Secondary VNIC approach
Adding a secondary VNIC is a straight forward means for granting Internet access to a Compute instance without affecting all Compute instances in a subnet. However, you should consider the following points before pursuing this model.
By default, a Compute instance with multiple VNICs will use the primary VNIC as the default gateway. Therefore, you must perform additional configuration (i.e., change the default gateway) in order to initiate Internet connections using the secondary VNIC. Alternatively, if you have the option to place the primary VNIC in the Public Subnet (using the example architecture above) and the secondary VNIC in the Private Subnet, you will find that your outbound Internet connectivity will work without additional configuration
For both Linux and Windows Compute instances, you must execute manual OS commands to activate/enable the secondary VNIC so that it is visible to the OS.
Online documentation references
Oracle documentation includes the steps to enable a secondary VNIC. Visit the links below for more details.
Additionally, for those with access to the Oracle Support Knowledge Base (which is available to all Oracle customers with an active Customer Service Identifier), see this Knowledge Base article that describes how to automatically activate the secondary VNIC on boot for Oracle Linux 6 and Oracle Linux 7.
The final option we will consider is to route the Internet-bound traffic through the OCI VPN or FastConnect service to the on-premises network for customer's using such a configuration. From there the outbound traffic may be handled using the customer's existing egress policies and NAT (if appropriate). Note, we will not cover the on-premises configuration as that will vary from customer to customer. Therefore, assume that the on-premises configuration is already set to provide outbound Internet access and we will discuss OCI configuration only.
First, ensure you have established a Dynamic Routing Gateway ("DRG") and are using that as your default gateway as shown here:
Next, ensure you have a functioning connection via your DRG to the on-premises network via a VPN tunnel or FastConnect.
In mid-November 2018, Oracle introduced "transit routing" as an enhancement to the routing capabilities in OCI. Using transit routing, it is now possible for multiple VCNs to share the same VPN or FastConnect connection. See this article in Oracle's online documentation for details how to implement transit routing:
What to consider when routing Internet connections through on-premises network
Consider the following when planning your Internet access solutions using on-premises routing.
If you are sending all outbound Internet connections as well as handling day-to-day traffic between the on-premises network and OCI you may find the latency for such a configuration unacceptable.
Likely the VPN/FastConnect connection to OCI has its own rules and routes in the on-premises configuration in order to properly segregate and manage traffic to and from OCI. Thus, you may have to introduce new configuration in your on-premises environment to provide properly-managed outbound Internet connectivity.
This is a minor point for well-configured FastConnect/VPN connections (i.e., with tunnel/circuit redundancy and so on). But, as described above related to using a NAT Instance, the NAT Gateway and Internet Gateway provide sound fault-tolerance and redundancy without any specific effort on your part. If you are using FastConnect/VPN as described here, ensure you have considered your availability requirements and tolerance for tunnel disruptions.
I hope you agree that there are multiple reasonable options in order to fulfill Internet connectivity needs for many different scenarios.
Oracle, as the Cloud Service Provider, makes updates on a regular basis. Because new OCI capabilities may come at any time, consider reviewing the "what's new" page regularly. Changes such as the NAT Gateway and Transit Routing (both noted above) were revealed here once generally available.
If you have any questions or need additional assistance, please contact the Mythics Cloud Team.
Jon-Eric Eliker, Cloud Solution Consultant, Mythics