How to build rpms

prepare the source file

first of all, you need all your source files ready under ~/RPMBUILD/SOURCES dircetory
For example if you wan to build jkd1.8.0 rpm, you may want the java8-local_policy.jar and java8-US_export_policy.jar from java 8 JCE and the jre-8u111-linux-x64.tar.gz from oracle website.

prepare the .spec file

once the source files are in place, you need to write a mybuild.spec file to instruct the rpm command how to pre, build, install, clean, files, preun, post the files in the rpm package.
this file has to be put in the ~/RPMBUILD/SPECS directory.

build rpm

now you can run the following command to build the rpm

rpmbuild -ba ~/RPMBUILD/SPECS/mybuild.spec

this command will put the resulting rpm in ~/RPMBUILD/RPMS/x86_64.

post to yum repo

You can optional set the rpm to be managed by yum.

Assuming you plan to place the repository in /var/www/html which is the default directory for Red Hat based Apache installations.

Also assuming you have run 

yum install yum-utils;

reposync -p /var/www/html/;

so that you can present this directory with apache.

now all you have to do is copy the rpm into /var/www/html/fedora/jdk8/

then run command

run createrepo /var/www/html/fedora/jdk8/

If your apache server is running, your new rpm should now be managed by yum.

If not, run it:

yum install httpd;

chekconfig httpd on;

service httpd start;

config yum client

On the client host you need to point the yum repo to the yum server you just setup,

for simplicity, lets use the yum server itself as client

create a /etc/yum.repos.d/local.repo file

[Local-Repository]

name=mylocal

baseurl=http://ip-of-my-server/fedora/jdk8

enabled=1

gpgcheck=0

final result

now once you type 

yum install rpm-name-set-in-spec-file

the rpm will be intalled

7 shell commands for network trouble-shooting

1. nslookup <url>
check what's your url is known by the dns server.

2. ping <IP>
Is the IP alive? If ping get 100% loss, it could mean two things, either the host is down or it is blocking the ICMP traffic.

3. curl <url>
check if the web application you are trying to access is alive.

4. netstat -nultp
ssh into the server and check what's going on there. run the command on the server machine to list all the listening ports of your application, for example, your java program's listening ports are:

netstat -nultp | grep java

5. nc <IP> <port>
Your web application might be listening on a port such as 8799, which you don't know how to communicate with. Use netcat, the general purpose net client to connect to the port to tell if the port is open.

6. openssl s_client -connect <IP:port>
Whenever your web application get authentication issue during network connection, get the public key from the destination server, then check whatever your web client was looking for are there.

7. tcpdump -i eth0 -s0 -v port 80
Still have issue? Try to get the pcap from the source host, load balancer, proxy server, destination host, then look into them, looking for issues such as connection reset, socket reuse, traffic drop, wait timed out, concurrency bug, etc.

How to consuming a soap endpoint in java

A soap end point looks something like
http://xyzcode.xyz/quote
usually the wsdl file can be download from
http://xyzcode.xyz/quote?wsdl

The wsdl file will give the instruction about how to consume the soap web service, that is all you needed to know in order to talk to soap api besides the user/pass for authentication.

Start with a wsdl file such as quote.wsdl, you can import the file into third-party gui tools such as soapUI, then you fill up all the fields for the request in xml format, and press send then wait, the soapUI will send your request to the remote endpoint with soap protocol, then get the response, then print the output in xml format.

Instead of manually filling the request fields in SoapUI GUI, you can have a java program to automatically fill all the needed fields, construct the request object, serialize it, have a SAAJ client send the request down to the wire with soap protocol, then wait for the SAAJ client to get the soap response back, then deserialize the response into response object.

The following tools and packages are what you need to make it happen.

1. apache wsdl2java.
Once you downloaded the tool from apache site, you can issue the following command to convert the wsdl file into corresponding java files.
The command is:
wsdl2java quote.wsdl

You most likely end up with 2 folder generated: com and org, within which, your wsdl generated java files reside.

An important thing to note is: the tool generated many ObjectFactory.java files for serializer and deserializer to consume. You can find the location of those OjectFactory.java by the following command:

find . -name ObjectFactory.java

the output could be something like:

./com/yourinventit/processing/android/serial/ObjectFactory.java
./com/yourinventit/processing/android/ObjectFactory.java
./org/helloexample/ObjectFactory.java

2. SAAJ client
The SOAP with Attachments API for Java or SAAJ provides a standard way to send XML documents over the Internet from the Java platform.

The maven saaj dependency is:

<dependency>
    <groupId>com.sun.xml.messaging.saaj</groupId>
    <artifactId>saaj-impl</artifactId>
    <version>1.5</version>
</dependency>

With saaj you can create soap calls like the following. In order to request, we need to create the Request message. Follow the same way we used to fill the request form in soapUI GUI, here we need to fill many fields in the request SOAPMessage object as well. We fill these fields with the help of wsdl generated classes.

SOAPConnectionFactory soapConnectionFactory = SOAPConnectionFactory.newInstance();
SOAPConnection connection = soapConnectionFactory.createConnection();
URL endpoint = new URL("http://xyzcode.xyz/quote");
MyPayload payload = business.getRequest();  //pass in from call param
SOAPMessage  message = funcCreateRequest(payload);  //use wsdl generated classes
SOAPMessage response = connection.call(message, endpoint);
connection.close();

3. jaxb
We said that we fill soap request fields with the help of wsdl generated classes. In fact, there are java library to help serialize and deserialize your soap request and response. The tool for that is jaxb.
You can include jaxb using maven.

<dependency>
    <groupId>javax.xml.bind</groupId>
    <artifactId>jaxb-api</artifactId>
    <version>2.2.12</version>
</dependency>
<dependency>
    <groupId>com.sun.xml.bind</groupId>
    <artifactId>jaxb-impl</artifactId>
    <version>2.2.11</version>
</dependency>

Once you have jaxb, you need to tell JAXBContext where to find the ObjectFactory.java files in order to serialize and deserialize soap objects generated by the wsdl file.
With the above example, your initialization will looks like:

JAXBContext jc = JAXBContext.newInstance("com.yourinventit.processing.android.serial:com.yourinventit.processing.android:org.helloexample:org.helloexample");

Once you have the JAXB initialized, you can convert between object and xml string.
For request

MyPayload payload = business.getRequest();  //pass in from call param
Marshaller m = jc.createMarshaller();
StringWriter w = new StringWriter();
xsw = XMLOutputFactory.newInstance().createXMLStreamWriter(w);
m.marshal(payload, xsw);  //consume objects specified in ObjectFactory.java
String requestString = w.getBuffer().toString();
SOAPMessage soapMessage = funcBuildSoapRequest(requestString);

For response
Node payload = soapMessage.getSOAPBody().getFirstChild();
String responseString  = funcConvertPayloadToString(payload);
Unmarshaller u = jc.createUnMarshaller();
xsr = XMLInputFactory.newInstance().createXMLStreamReader(new StringReader(responseString));
Object o = unmarshaller.unmarshal(xsr);  //consume objects specified in ObjectFactory

The "Object o" can be cast to one of the wsdl object, then you can pass the object around in your program.

7 tools that monitor your servers and web applications

Your web application is a distributed software system that runs 24/7 on a fleets of servers. There are many functional components such as DNS server, load balancer, router, web server, authentication service, provisioning service, redex server, database server etc. Constantly they are generating data such as request count, response time, cpu load, ram usage, disk usage, web container thread pool size, application thread pool size, database connection pool size, garbage collection time, etc. For each data source, you can get static data such as average, sum, min, max, perc99, perc95, etc. You can then sort them by site, hostname, IP, date, tag, source etc. Sometimes, there are correlations between data types. For example high request count often correlates to high cpu load, high ram, high thread-pool size and longer database response time, etc.

The following 7 tools will help you to collect the data from your application and organize them in smart ways, so that your machines are under control.

1. Event queues

Your web application is a group of applications. Each application is deployed on a fleet of servers behind load balancer, and each copy of the artifact deployed on each server is a multi-threading program. As you can tell, there are high chance for race conditions. We don't want our application meta data collection ruin the main functionality or create synchronize bottleneck. A thread-safe queue implementation can help a lot in this situation. It decouples producer and consumer, so that the log generated from one thread in one VM won't mess up with another thread in the same VM or another VM. For example Kafka and RabbitMQ are two of this kind.  Your application can send events as object to a topic/stream of the queue. These topics, after processing, can be sent to the API of consumers such as another app, database, zabbix, splunk, tableau etc.

2. Data analytic tools

Your Kafka stream can be sent to data analytic tools such as splunk. Once the data get there, they are stored in the way fast retrieving and sorting are possible. Splunk can analyze huge amount of data, use them to create statistics, time charts and dashboards. Splunk also allows events to trigger alerts. The alert threshold criteria can range from count larger than a number to time longer than a value. Once the alert is triggered, actions can be taken, for example, send message to slack channel, email, SMS, ticketing API.

3. Incident Response Platform

There are many ticketing platforms such as bmc remedy, salesforce, pagerduty. Pagerduty for example, allows you to install an app in splunk. One of the splunk alert action is send to a pagerduty service. Once the pagerduty service receives a call, it acts according to the escalation policy -- for example, call the primary contacts, if not get acknowledgement within 5 minutes, call the secondary contacts, so on and so forth, until someone takes action about the event-- could be annoying during night hours.

4. Host inventory tools

Your server warehouse need management. Hardware as well as VMs can be managed with tools such as zabbix. Once the zabbix agent is installed on an unix server, it starts to collect infrastructure informations such as cpu, memory, hdc io, hdc bw, sda io, sda bw, etc. It allows monitor, inventory and report about infrastructure hosts on the Zabbix server. You can also draw graph with your hardware resource usage history.

5. VM and cloud inventory tools

Virtual machines and cloud VM instances are special resources. They are special because they are elastic and volatile. Tools such as VMWare Vcenter/Vsphere, AWS, GCP, AZURE allows you to quickly create/destroy VMs, create cloud virtual company, reconfigure routing rule, setup ACL, configure scale up/down policy etc. Oracle has its own JVM debug tool which can be invoked with command JVisualVM.

6. Data warehouse mining and visualization tools

Tools such as tableau can connect to almost any data warehouse applications on the market: oracle, mysql, AWS Redshift, cubes, Teradata, cassandra, data lake, redis, microsoft SQL Server, mongodb, hyperion, etc. With the wealth of data already stored in the data warehouse, tableau can generate reports, statistics, graphs across different data sources and give the user power to further analyze them.

7. Network inventory tool

Your network resources such as IPs, Nodes, Servers need to be organized and grouped. There are tools such as NGINX, BIG IP to help you out. At anytime, you are able to inventory your network resource by locations, OS, liveness, cells etc. You are also able to take a group of servers out of service or put them into service.

tools for handling multiple ssh on mac os x

If you ever worked in cooperate environment or be the network administrator, you sometimes need to login many servers and check stuff simultaneously.

On windows, you can use putty or smart putty to handle multiple ssh connections.

Here are some tools for mac os x equivalent.

csshx

installation:

brew install csshx

usage:

csshx --login username host1 host2 host3 host4

xterm

installation:

download the mac version from the xterm website www.iterm2.com 

usage:

add a connection profiles:

click Profiles -> Open Profiles... -> Edit Profiles -> click the + sign to add a list of hostnames and ip addresses you would like to connect to -> when creating a host record, select command radio button then type in ssh command.

open multiple profiles simultaneously in tabs:

click Profiles -> Open Profiles... -> use shift or control key to hightlight a list of hosts you want to connect to -> click New Tab

The highlighted hosts will be connected and open in tabs.

send commands to multiple tabs:

click Shell -> Broadcast Input -> Broadcast Input to All Panels in All Tabs

7 useful google search syntax

We use google search all the time, mostly by typing a keyword into the search bar and press enter. This gives us lots of content, sometimes more than needed. With some constrain, we can narrow down the search.

1. "java +xyzcode" this will search java, but the page must contains xyzcode, you can change + to - to indicate that the page must not contain xyzcode.
2. "cache:xyzcode.blogspot.com" this will search google cached page history. As a blogger, if you saved something by mistake, don't panic, google has the backup. 
3. "site:xyzcode.blogspot.com java" this will search java only on website xyzcode.blogspot.com
4. "filetype:pom JGraphT" this will search JGraphT only in file type pom.
5. "inurl:xyzcode" this will search xyzcode in the url itself.
6. "xyzcode@facebook" this will search xyzcode in facebook.
7. "#xyzcode" this will search hashtag xyzcode.

7 steps to allow you access your mac os x from internet

You can ssh into your home mac computer while traveling. There are a few steps to do:

step 1, go to system preferences, check sharing, check remote login. Only allow the unprivileged user to remote login, remove admin from the allowed list.

remote login enable

step 2. go to system preferences -> users & groups select the user you need to remote access the computer, change the password to be a very strong one.

step 3. if your firewall is blocking port 22, enable it. Go to system preferences -> Security and Privacy -> Firewall Options.

step 4. schedule wake up, if your computer go to sleep you won't be able to ssh into it. Go to system preferences -> Energy saver -> Schedule

schedule wakeup

step 5.  test remote login from LAN address, ssh <user>@<localIp>.

step 6. login your gateway router, add a port forwarding rule, forward port 22 to the ip address of your computer.

port forwarding

step 7. test remote login from internet address. The internet address can be found from gateway router.

You can disable password login and only allow certificate based login, but the above 7 steps should be able get your WAN access goal reached.

7 Mac OS X tools for web application trouble-shooting

Web applications use server-client pair to communicate via open sockets. In order to trouble-shooting them, we sometimes need to trace requests through computer system. We need to post request to web service end-point and analyze the returned xml, json etc. We sometimes need to check the server logs, databases, splunk, etc. Here are some tools to make these job easy.

TextMate

The software TextMate allows you to format text so that they are easy to read. Json, for example, is intended to be compact, so chances are the json string passed in web application request/response have no space or line break. In order to review the content, you need to format the compact string so that space and line break will be inserted for easy reading. You can copy the json string from a log and paste it into textmate then select json format to get an easy to read version. You can do the same for compact xml string, html string etc.

PostMan

Whenever you need to test rest services, you can use curl commands. However, with PostMan, your life will be easier. This tool allows you to easily construct the request body, set headers, credentials, etc. You can group and save various rest service requests for later use. The request and response are formatted too, sweet.

Iterm

Terminal is a basic scripting environment in mac os x, Iterm is a much better alternative. You can do many cool things such as sending commands to multiple tabs, script login session, split windows, highlight text to copy to clipboard, use many shot-cuts, etc.

Mysql Workbench

This tool manages your mysql database connections. You can browse schema, run query, check result, copy results, export results, analyze performance etc.

Oracle SQL Developer

a free tool to mange oracle database connections. UI provides many features such as browsing schema, view single record, syntax highlighting etc.

Slack

A computer system is developed by a team, so trouble-shooting often involves a team of engineers. Slack allows you to chat with partner, host online meetings across the internet. A meeting channel recorded the trouble-shooting history for later reference.

Notes

Last but not least, organize your thoughts before and after communication. Mac OS X notes is the handy tool for you.

7 steps to secure linux

Linux/Unix have many species, here we talk about those species designed for efficiency and security instead of luxury and convenience.

Nowadays, the only safe computer system is the dead brick. The one has no network connections -- wifi, bluetooth, cable, etc. You can only interact with it by physically sitting in front of it and typing on the keyboard.

step 1. get the trusted linux/unix distribution. 

Since most linux are open source, that means everybody have access to the source code and can modify it. You want to make sure you get the trusted distribution. So never download the Linux images from anywhere other than the official sources. Always be sure to verify the SHA256 checksums of the file you’ve downloaded against the official values. It would be easy for a malicious entity to modify a installation to contain exploits or malware and host it unofficially.

step 2. set a complex password for root

Without guarding root access, any security hardening is a waste of time.

step 3. boot into dead brick.

A runlevel is one of the modes that a Unix -based operating system will run in. Each runlevel has a certain number of services stopped or started, giving the user control over the behavior of the machine.

During the boot process for Redhat 9.0 and Fedora Core systems, for example, a sample /etc/inittab file defines the runlevel as follows:

# Default runlevel. The runlevels used by RHS are:
#   0 - halt (Do NOT set initdefault to this)
#   1 - Single user mode
#   2 - Multiuser, without NFS (The same as 3, if you do not have networking)
#   3 - Full multiuser mode
#   4 - unused
#   5 - X11
#   6 - reboot (Do NOT set initdefault to this)
#
id:2:initdefault:

This tells the init process that the default run level for the system is run level 2. This runlevel disables network access, solely use command line without the overhead of X11 based GUI.

step 4. disabling linux services

Now you have a safe dead brick, you can take time to disable any services that you don't actually need which expose extra access ports into your linux server if you leave them running in the background.

On redhat, to list all service settings run the following command:

    /sbin/chkconfig --list

This will display a long list of services showing whether or not they are started up at various runlevels. An example line looks like:

    httpd           0:off   1:off   2:off   3:on    4:off   5:on    6:off

chkconfig can also be used to change the settings. If we wanted the HTTP service not to start up when we at runlevel 5 we would issue the following command:

    /sbin/chkconfig --level 5 httpd off

on the other hand, you want to enable syslog
   /sbin/chkconfig --level 235 syslog on

step 5. set up firewall

Software firewall on your linux box is the second line of defense for your linux system, the main defense is the hardware firewall on your network gateway. So if you are in a dangerous environment, such as in shared public network, your software firewall is the only defense for your linux system.

The following command list all the firewall rules defined by your iptable:
iptables -L -v

You can start by the most restrict rule, then open some connection for the ones you know for sure.
The following rules deny all connections except a one way connection to ip 10.10.10.10 on port 80. HTTP connections TO 10.10.10.10 are permitted, but HTTP connections FROM 10.10.10.10 are not. However, the system is permitted to send back information over HTTP as long as the session has already been established.

iptables --policy INPUT DROP
iptables --policy OUTPUT DROP
iptables --policy FORWARD DROP
iptables -A OUTPUT -p http --dport ssh -s 10.10.10.10 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp --sport 80 -d 10.10.10.10 -m state --state ESTABLISHED -j ACCEPT

step 6. Update the OS

Keep the OS updated so that you get the latest security patches.

step 7. resist the temptation of installing unofficial applications

The single most common causes of a broken Linux installation are following unofficial advice, and particularly arbitrarily installing softwares from unofficial repositories.

7 good habits for securing your windows 7

Old laptops running windows 7 are sitting there catching dust. Depends on your style, you can trade them with the latest cutting edge PC, or you can get value out of them with memory upgrades and proper usage. Besides, if some installed softwares have to be run on old windows platform like windows 7, upgrading the operation system won't be an option.

windows 7

Old operation system pose security risks to you network. The vendor is less actively support these branches, so bug fixes are released less frequently. The bright side is, hackers are less interested in these old systems as well, so less dark energy is dedicated to explore holes in these old platforms. As a result, both defender side and attacker side are less interested in these old gears, they just move into to those new lubricate battle fields.

Defending windows in-depth sometimes do need PHD degree, however, with good habits, we can still practically put these old but not obsolete windows 7 boxes into use.

1. Use normal account instead of admin account for your everyday work, set a strong password.
2. enable automatic windows update, make sure the latest bug fixes and system enhancement are installed . Microsoft is the key player on defending windows operation system. As long as vulnerabilities are found on windows, microsoft will keep patching the existing operation system. These patching are critical to keep your PC safe from malwares. In order to control your computer, malware have to gain privileges in order to run command line/shellcode to install payload. Some sort of system bug has to be there to aid the privilege escalation, either some buggy code allowing buffer overflow or sql injection to reveal use/pass of admin from database, etc. Windows updates fix those buggy code which the hackers are looking for.
3. Avoid using IE, uninstall active X components, use other browsers like chrome or firefox instead. It sounds mean to microsoft, however, active X might be one of the major reason microsoft is called "evil" in popular culture. Hackers are working hard to gain privilege to run malicious code in order to install malwares, windows's active X give them such privilege for free. As long as windows found needed active X components, IE are allowed to run codes and install programs on the host in the background without asking for permission. This opened door for drive by infection -- just by browsing a webpage with malware content, your IE can download and install them with the aid of active X without your notice. Microsoft itself stopped using active X in Edge browser, which is the replacement of IE. Unfortunately, at the time of this post, Edge browser haven't been ported from windows 10 to windows 7, so other browsers like chrome and firefox is better on windows 7 from the perspective of security.
4. Use Windows Defender to protect against spyware and potentially unwanted software, keep the windows defender up to date. Other choices are third party anti-virus (anti-malware) software like Norton. You can use windows defender alone or use both (risking conflict), the bottom line is you have to have one. These anti-malware software find malware signature by scanning your computer and catch them. Windows Defender used to be scored much lower than its competitors, but since windows 10, it has caught up.  Microsoft's own anti-malware product windows defender has the advantage of being free and intimate to windows, which is a proprietary operation system. It might have better chance to detect rootkit since it knows the windows source code.
5. enable windows firewall. Windows firewall can help prevent hackers or malicious software from gaining access to your computer through the internet or a network. 
6. Only install softwares from reputable source. The difference between bad softwares and malwares are just you perception. A non-professional developer can write a program that provides bugs for hacker to explore, it can slow down your computer by consuming too much resources, it can refuse to be uninstalled, or even sending your sensitive data somewhere out to the internet. Blocking rogue applications' inbound and outbound traffic with windows firewall rules can prevent them from ringing home before we find way to erase them from disk.
7. Avoid visiting dangerous websites. If you have to visit them, use guest account instead.

Enable firewall logs on MacOs High Sierra

Mac OS X v10.5.1 and later include an application firewall you can use to control connections on a per-application basis (rather than a per-port basis). This makes it easier to gain the benefits of firewall protection, and helps prevent undesirable apps from taking control of network ports open for legitimate apps.

application firewall

To enable it, go to System Preferences -> Security & Privacy -> Firewall
You can choose a few firewall options. The most secure or restricted option is block all incoming connections. With this option selected, hackers on the wild can not connect to your computer, they can not even discover your existance.

However, if you have malware already installed on your computer, such as a keylogger, Adware, backdoor, (in practice, this kind of malware is rare on Mac OS X, but there is no guarantee the landscape won't change in the future), even blocking all incoming connections won't help here. The keylogger/adware/backdoor will initialize an outcoming connection to give away your sensitive data, which the firewall won't block. These out bounding traffic are generally small, easy to hide in normal traffic such as your web browsing traffic.

Installing an expensive IDS/IPS/UTM device in your home network is the ultimate solution. However, if you have some experience with stateful firewall, you can detect the suspicious outbound traffic by reviewing and searching firewall logs on both the end-point and the gateway (some oddness can easily stand out, like mid-night traffic.). IDS/IPS/UTM in essence are collection of searching and matching actions in automation.

By default, the firewall log on Mac Os High Sierra is empty, this is because even after you turns on firewall which enables log, the firewall log option is throttled. You have to change the default settings from throttle to detail or brief.

/usr/libexec/ApplicationFirewall/socketfilterfw --getloggingopt

sudo /usr/libexec/ApplicationFirewall/socketfilterfw --setloggingopt detail

Once the default settings are changed, you can view the firewall logs either from command line

tail -F /var/log/appfirewall.log

application firewall log

or from Finder -> Application -> Utilities -> Console.

ifconfig output on MacOs High Sierra

In a typical Mac OS X, type ifconfig in command line will give a long list of interfaces.

network>ifconfig

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384

options=1203<RXCSUM,TXCSUM,TXSTATUS,SW_TIMESTAMP>

inet 127.0.0.1 netmask 0xff000000 

inet6 ::1 prefixlen 128 

inet6 xxx prefixlen 64 scopeid 0x1 

nd6 options=201<PERFORMNUD,DAD>

gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280

stf0: flags=0<> mtu 1280

EHC29: flags=0<> mtu 0

EHC26: flags=0<> mtu 0

XHC20: flags=0<> mtu 0

en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500

options=10b<RXCSUM,TXCSUM,VLAN_HWTAGGING,AV>

ether xxx 

nd6 options=201<PERFORMNUD,DAD>

media: autoselect (none)

status: inactive

en1: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500

ether xxx 

inet6 xxx prefixlen 64 secured scopeid 0x8 

inet xxx netmask 0xffffff00 broadcast 192.168.2.255

nd6 options=201<PERFORMNUD,DAD>

media: autoselect

status: active

en2: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500

options=60<TSO4,TSO6>

ether xxx 

media: autoselect <full-duplex>

status: inactive

fw0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 4078

lladdr xxx 

nd6 options=201<PERFORMNUD,DAD>

media: autoselect <full-duplex>

status: inactive

p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304

ether xxx 

media: autoselect

status: inactive

awdl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1484

ether xxx 

inet6 xxx%awdl0 prefixlen 64 scopeid 0xc 

nd6 options=201<PERFORMNUD,DAD>

media: autoselect

status: active

bridge0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500

options=63<RXCSUM,TXCSUM,TSO4,TSO6>

ether xxx 

Configuration:

id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0

maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200

root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0

ipfilter disabled flags 0x2

member: en2 flags=3<LEARNING,DISCOVER>

        ifmaxaddr 0 port 9 priority 0 path cost 0

nd6 options=201<PERFORMNUD,DAD>

media: <unknown type>

status: inactive

utun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 2000

inet6 xxx%utun0 prefixlen 64 scopeid 0xe 

nd6 options=201<PERFORMNUD,DAD>

network>

The following are the explanations about these interfaces:


lo0 is the loopback device, which is used for entirely internal communication such as between two applications running on localhost.

gif0 The gif interface is a generic tunnelling device for IPv4 and IPv6. It can tunnel IPv[46] traffic over IPv[46]. Therefore, there can be four possible configurations. The behavior of gif is mainly based on RFC2893 IPv6-over-IPv4 configured tunnel. It is the mac os X default interface, not a security concern.

6to4 interfaces

stf0 is SixToFour interface. 6to4 is an Internet transition mechanism for migrating from Internet Protocol version 4 (IPv4) to version 6 (IPv6), a system that allows IPv6 packets to be transmitted over an IPv4 network (generally the IPv4 Internet) without the need to configure explicit tunnels. Special relay servers are also in place that allow 6to4 networks to communicate with native IPv6 networks. It is a default interface, not a security concern.

EHC29
EHC26
XHC20
With macOS High Sierra you can use Wireshark to capture USB traffic.  The interface needs to be manually brought up/down to enable/disable packet capture for the specific controller via ifconfig. The capture interfaces are named based on the underlying controller type followed by the bus number:

$ ifconfig
EHC26: flags=0<> mtu 0
XHC20: flags=0<> mtu 0
EHC29: flags=0<> mtu 0

$ ioreg -w0 -rc AppleUSBHostController
+-o XHC1@1400
+-o EHC2@1a00
+-o EHC1@1d00

The format above is @ where the most significant byte of the location is the bus number.  For example, if the device your interested in is connected to the XHCI controller XHC1@1400 then you would enable packet capture via "sudo ifconfig XHC20 up” and disable via “sudo ifconfig XHC20
down”.  Once the interface is up then Wireshark will be able to capture/decode/filter USB traffic for that controller.
These interfaces are MacOs High Sierra default, not security concern.

en0
en1
en2
physical network interfaces. Typically, one of them is the Ethernet interface, one of them is the Airport wifi interface, one of them is the bluetooth interface.

Firewire port

fw0 is networking over firewire. You can connect a Firewire cable between two Macs, and OS X can use that cable as a network connection.

Mac thunderbolt port

bridge0 is thunderbolt bridge. You can connect two Thunderbolt-equipped Mac computers using a Thunderbolt cable, then use internet protocol to communicate between the computers.

PPP (PPPSerial)

p2p0 peer to peer serial connection interfaces. If your MacOs have been connected to arduino through USB port, you will have this entry.

awdl0 AWDL (Apple Wireless Direct Link) is a low latency/high speed WiFi peer-to peer-connection Apple uses for everywhere you’d expect: AirDrop, GameKit (which also uses Bluetooth), AirPlay, and perhaps elsewhere. It works using its own dedicated network interface, typically “awdl0". By having multiple interfaces, Apple is able to have your standard WiFi connection on en*, while still broadcasting, browsing, and resolving peer to peer connections on awdl0.

oracle VirtualBox

utun0 it is the tunnel interface. If you have VirtualBox installed, guest operation system in virtualBox will use this interface to communicate with host operation system. TUN (namely network TUNnel) simulates a network layer device and it operates with layer 3 packets like IP packets. TUN is used with routing. Packets sent by an operating system via a TUN device are delivered to a user-space program which attaches itself to the device. A user-space program may also pass packets into a TUN device. In this case the TUN device delivers (or "injects") these packets to the operating-system network stack thus emulating their reception from an external source.

vment* - is used by VMWare Fusion to provide networking to your virtual machines, and there's likely to be one per VM you have set up.

ports and protocols on Mac OS X

mac os x use Darwin unix at the core, which is heavily influenced by BSD. The following is the ports.

network>cat /etc/protocols

#

# Internet protocols

#

# $FreeBSD$

# from: @(#)protocols 5.1 (Berkeley) 4/17/89

#

# See also http://www.iana.org/assignments/protocol-numbers

#

ip 0 IP # internet protocol, pseudo protocol number

#hopopt 0 HOPOPT # hop-by-hop options for ipv6

icmp 1 ICMP # internet control message protocol

igmp 2 IGMP # internet group management protocol

ggp 3 GGP # gateway-gateway protocol

ipencap 4 IP-ENCAP # IP encapsulated in IP (officially ``IP'')

st2 5 ST2 # ST2 datagram mode (RFC 1819) (officially ``ST'')

tcp 6 TCP # transmission control protocol

cbt 7 CBT # CBT, Tony Ballardie <A.Ballardie@cs.ucl.ac.uk>

egp 8 EGP # exterior gateway protocol

igp 9 IGP # any private interior gateway (Cisco: for IGRP)

bbn-rcc 10 BBN-RCC-MON # BBN RCC Monitoring

nvp 11 NVP-II # Network Voice Protocol

pup 12 PUP # PARC universal packet protocol

argus 13 ARGUS # ARGUS

emcon 14 EMCON # EMCON

xnet 15 XNET # Cross Net Debugger

chaos 16 CHAOS # Chaos

udp 17 UDP # user datagram protocol

mux 18 MUX # Multiplexing protocol

dcn 19 DCN-MEAS # DCN Measurement Subsystems

hmp 20 HMP # host monitoring protocol

prm 21 PRM # packet radio measurement protocol

xns-idp 22 XNS-IDP # Xerox NS IDP

trunk-1 23 TRUNK-1 # Trunk-1

trunk-2 24 TRUNK-2 # Trunk-2

leaf-1 25 LEAF-1 # Leaf-1

leaf-2 26 LEAF-2 # Leaf-2

rdp 27 RDP # "reliable datagram" protocol

irtp 28 IRTP # Internet Reliable Transaction Protocol

iso-tp4 29 ISO-TP4 # ISO Transport Protocol Class 4

netblt 30 NETBLT # Bulk Data Transfer Protocol

mfe-nsp 31 MFE-NSP # MFE Network Services Protocol

merit-inp 32 MERIT-INP # MERIT Internodal Protocol

dccp 33 DCCP # Datagram Congestion Control Protocol

3pc 34 3PC # Third Party Connect Protocol

idpr 35 IDPR # Inter-Domain Policy Routing Protocol

xtp 36 XTP # Xpress Tranfer Protocol

ddp 37 DDP # Datagram Delivery Protocol

idpr-cmtp 38 IDPR-CMTP # IDPR Control Message Transport Proto

tp++ 39 TP++ # TP++ Transport Protocol

il 40 IL # IL Transport Protocol

ipv6 41 IPV6 # ipv6

sdrp 42 SDRP # Source Demand Routing Protocol

ipv6-route 43 IPV6-ROUTE # routing header for ipv6

ipv6-frag 44 IPV6-FRAG # fragment header for ipv6

idrp 45 IDRP # Inter-Domain Routing Protocol

rsvp 46 RSVP # Resource ReSerVation Protocol

gre 47 GRE # Generic Routing Encapsulation

dsr 48 DSR # Dynamic Source Routing Protocol

bna 49 BNA # BNA

esp 50 ESP # encapsulating security payload

ah 51 AH # authentication header

i-nlsp 52 I-NLSP # Integrated Net Layer Security TUBA

swipe 53 SWIPE # IP with Encryption

narp 54 NARP # NBMA Address Resolution Protocol

mobile 55 MOBILE # IP Mobility

tlsp 56 TLSP # Transport Layer Security Protocol

skip 57 SKIP # SKIP

ipv6-icmp 58 IPV6-ICMP icmp6 # ICMP for IPv6

ipv6-nonxt 59 IPV6-NONXT # no next header for ipv6

ipv6-opts 60 IPV6-OPTS # destination options for ipv6

# 61 # any host internal protocol

cftp 62 CFTP # CFTP

# 63 # any local network

sat-expak 64 SAT-EXPAK # SATNET and Backroom EXPAK

kryptolan 65 KRYPTOLAN # Kryptolan

rvd 66 RVD # MIT Remote Virtual Disk Protocol

ippc 67 IPPC # Internet Pluribus Packet Core

# 68 # any distributed filesystem

sat-mon 69 SAT-MON # SATNET Monitoring

visa 70 VISA # VISA Protocol

ipcv 71 IPCV # Internet Packet Core Utility

cpnx 72 CPNX # Computer Protocol Network Executive

cphb 73 CPHB # Computer Protocol Heart Beat

wsn 74 WSN # Wang Span Network

pvp 75 PVP # Packet Video Protocol

br-sat-mon 76 BR-SAT-MON # Backroom SATNET Monitoring

sun-nd 77 SUN-ND # SUN ND PROTOCOL-Temporary

wb-mon 78 WB-MON # WIDEBAND Monitoring

wb-expak 79 WB-EXPAK # WIDEBAND EXPAK

iso-ip 80 ISO-IP # ISO Internet Protocol

vmtp 81 VMTP # Versatile Message Transport

secure-vmtp 82 SECURE-VMTP # SECURE-VMTP

vines 83 VINES # VINES

ttp 84 TTP # TTP

#iptm 84 IPTM # Protocol Internet Protocol Traffic

nsfnet-igp 85 NSFNET-IGP # NSFNET-IGP

dgp 86 DGP # Dissimilar Gateway Protocol

tcf 87 TCF # TCF

eigrp 88 EIGRP # Enhanced Interior Routing Protocol (Cisco)

ospf 89 OSPFIGP # Open Shortest Path First IGP

sprite-rpc 90 Sprite-RPC # Sprite RPC Protocol

larp 91 LARP # Locus Address Resolution Protocol

mtp 92 MTP # Multicast Transport Protocol

ax.25 93 AX.25 # AX.25 Frames

ipip 94 IPIP # Yet Another IP encapsulation

micp 95 MICP # Mobile Internetworking Control Pro.

scc-sp 96 SCC-SP # Semaphore Communications Sec. Pro.

etherip 97 ETHERIP # Ethernet-within-IP Encapsulation

encap 98 ENCAP # Yet Another IP encapsulation

# 99 # any private encryption scheme

gmtp 100 GMTP # GMTP

ifmp 101 IFMP # Ipsilon Flow Management Protocol

pnni 102 PNNI # PNNI over IP

pim 103 PIM # Protocol Independent Multicast

aris 104 ARIS # ARIS

scps 105 SCPS # SCPS

qnx 106 QNX # QNX

a/n 107 A/N # Active Networks

ipcomp 108 IPComp # IP Payload Compression Protocol

snp 109 SNP # Sitara Networks Protocol

compaq-peer 110 Compaq-Peer # Compaq Peer Protocol

ipx-in-ip 111 IPX-in-IP # IPX in IP

carp 112 CARP vrrp # Common Address Redundancy Protocol

pgm 113 PGM # PGM Reliable Transport Protocol

# 114 # any 0-hop protocol

l2tp 115 L2TP # Layer Two Tunneling Protocol

ddx 116 DDX # D-II Data Exchange

iatp 117 IATP # Interactive Agent Transfer Protocol

stp 118 STP # Schedule Transfer Protocol

srp 119 SRP # SpectraLink Radio Protocol

uti 120 UTI # UTI

smp 121 SMP # Simple Message Protocol

sm 122 SM # SM

ptp 123 PTP # Performance Transparency Protocol

isis 124 ISIS # ISIS over IPv4

fire 125 FIRE

crtp 126 CRTP # Combat Radio Transport Protocol

crudp 127 CRUDP # Combat Radio User Datagram

sscopmce 128 SSCOPMCE

iplt 129 IPLT

sps 130 SPS # Secure Packet Shield

pipe 131 PIPE # Private IP Encapsulation within IP

sctp 132 SCTP # Stream Control Transmission Protocol

fc 133 FC # Fibre Channel

rsvp-e2e-ignore 134 RSVP-E2E-IGNORE # Aggregation of RSVP for IP reservations

mobility-header 135 Mobility-Header # Mobility Support in IPv6

udplite 136 UDPLite # The UDP-Lite Protocol

mpls-in-ip 137 MPLS-IN-IP # Encapsulating MPLS in IP

manet 138 MANET # MANET Protocols (RFC5498)

hip 139 HIP # Host Identity Protocol (RFC5201)

shim6 140 SHIM6 # Shim6 Protocol (RFC5533)

wesp 141 WESP # Wrapped Encapsulating Security Payload (RFC5840)

rohc 142 ROHC # Robust Header Compression (RFC5858)

# 138-254 # Unassigned

pfsync 240 PFSYNC # PF Synchronization

# 253-254 # Use for experimentation and testing (RFC3692)

# 255 # Reserved

divert 258 DIVERT # Divert pseudo-protocol [non IANA]

network>