Resources

March 18 2022

Cybersecurity Reporting Act Ratified

Mike Stute Security, Trusted Advisor cybersecurity, reporting

On Tuesday, as part of the omnibus bill, President Biden signed into law the Cybersecurity Reporting Act. This act requires companies operating in critical infrastructure to report to the Cybersecurity and Infrastructure Security Agency (CISA) within 72 hours after they discover they are under cyberattack. It also requires these same companies to report to the CISA within 24 hours if they choose to make a payment due to ransomware. This is an effort by the federal government to remove the plausible deniability of these companies regarding cyberattacks. Like many companies, these corporations choose to under invest in cybersecurity or rely on cybersecurity insurance to avoid investing in better cybersecurity. This has been prompted by the difference between reported incidents in 2021, amounting to about $29 million, and the much larger amount discovered by Chainalysis Inc. (which tracks cryptocurrency transactions) of $406 million (see Ransomware 2021: Critical Mid-year Update.)

Cyber Security Solutions

What It Means

Several line items on this new law have changed the game completely.

The CISA has the authority to subpoena companies that fail to report cybersecurity incidents or ransomware payments. Failure to comply with the subpoena can result in the case being sent to the Department of Justice.
Companies will need to show they are taking cybersecurity more seriously. Those without a CISO will probably need to hire one, and give the CISO veto over the CIO regarding cybersecurity– related decisions.

In return the CISA is required to warn organizations of vulnerabilities that ransomware actors exploit. The director of CISA (currently Jen Easterly) will need to establish a joint ransomware task force to coordinate federal efforts working with the industry to prevent and disrupt ransomware attacks.

The sixteen critical infrastructure sectors are chemical, communications, dams, emergency services, financial services, government facilities, information technology, transportation, commercial facilities, critical manufacturing, defense industrial, energy, food and agriculture, healthcare and public health, nuclear reactors (including materials and waste), and water and wastewater systems. For more information about what is included in each sector see (CISA – Critical Infrastructure Sectors).

January 13 2022

Time to Check with Your EDR Vendor

Mike Stute Resources, Security EDR, security

This month (January 2022), a team of researchers at the University of Piraeus, working in the informatics department, devised, and ran a series of tests to determine the state of Endpoint Detection/Response (EDR) systems as a follow up to a previously released paper. The researchers chose four attack methods that are found in the “wild,” meaning those that are currently being used by attackers. Because most intrusions start with a spear phishing attack where a user is lured to click on a malicious payload attached to an E-mail, the researchers chose this as their attack vector. This is quite common and the EDR should check the payload to block it before it can do harm, or at the very least, recognize the system has been comprised and report it.

The four vectors were

An executable DLL in a Control Panel applet, commonly known as a “cpl” file. This does not look like an executable but is loaded and run by a delegate to try to hide it from the EDR client.
A DLL that a MS Teams installation will load. The code will use a self-injection technique using an offensive toolkit called AQUARMOURY Brownie. This allows the malicious code to be executed in the address space of a “trusted” executable which means the EDR system may not see it.
A PE EXE file that uses another technique in the AQUARMOURY toolkit called “Early Bird” to inject the malicious process in an evasive method.
An HTA file which is a type of file that is a collection of code (hypertext, VBScript, JScript) that is run as part of a web page (or as an E-mail attachment) and executed by the Microsoft HTML Application Host (mshta.exe). This is yet another technique of code injection where trusted address space is used to hide execution from an EDR.

Looking For IT Solutions?

Again, these techniques are all common in toolkits (such as AQUARMOURY) to evade EDR detection. Since they are well known they should be detected by EDR vendors that are maintaining their code base to counter modern techniques. The results are quite telling.

Source: “An Empirical Assessment of Endpoint Security Systems Against Advanced Persistent Threats Attack Vectors”

In the far-left column is the product being tested which includes both EDR and EDP systems. The next four columns show each of the attack vectors chosen for the study. An X indicates the attack failed and that alerts were raised. A checkmark indicates the attack was successful without any alert raised. All vendors but two allowed the malicious code to execute in one or more of the attack vectors. Only one allowed all four vectors to work while most of the others allowed at least one, with the majority being two or more. Only FortiEDR and Sentinel One (all features turned on) stopped all four attacks.

After the initial test, the researchers used what they had learned to try to disable EDR clients. These techniques included unlinking client callback functions, manually patching vector tables or functions used to gather telemetry or using known vulnerabilities to disable or kill the EDR client. Once again, these techniques used known methods and offensive toolkits already available. The extended research was able to disable or kill MS Defender for Endpoints, Sophos Intercept X, Bitdefender, and FortiEDR. Since FortiEDR was one of the two systems that detected all attacks, but could be disabled by a kill technique, only Sentinel One survived intact without any fails, provided all features are enabled.

With these results in hand, you should check with your vendor to determine what steps they have taken to mitigate these attacks. As with all security layers, nothing is hack proof. The strength of a security system is its well-chosen layers of defense to work together to defend a system instead of relying on a single element. However, this does show a difference in the quality of EDR systems now. As companies respond these results will change but, in the end, it shows what all security researchers know: techniques exist to evade tools trying to find them. It is in the holistic security system layers that real security exists.

Questions? Please feel free to contact me about these results or a review of your own security defensive posture. View the research paper.

March 30 2021

Getting Ready for Round Three

Mike Stute Resources, Security

In the wake of the Solar Winds upstream attack we find ourselves in the midst of the ProxyLogon attacks on Microsoft Exchange servers. Both of these have a high penetration in their field of victims (18,000 for Solar Winds and 400,000 for ProxyLogon). But there’s more to come and the third round is going to be even bigger.

SickCodes released a vulnerability it found on the 28th (yes, March 28th, 2021) it found in the netmask package which is part of the NPM registry. NPM stands for “Node Package Manager” which is the ubiquitous (almost universal) package manager for Node.js. Node.js itself is a framework that creates an asynchronous event-driven JavaScript runtime which is heavily used in web development. In fact, according to https://hostingtribunal.com/blog/node-js-stats/#gref there are more than 20 million websites using it, more than 1 billion downloads with 6.3 million websites in the U.S alone. Some companies that rely on Node.js are Amazon, CitiBank, Netflix, eBay, Reddit, LinkedIn, Tumblr, PayPal, Wal-Mart, and Uber but these are just the biggest ones. The vulnerability leads to server side request forgeries which leads to a whole host of attack vectors including being able to force the web server to make connections to any server the attacker chooses. The worst case is the attacker using the web server to make connections to the internal servers to which it has access.

Because Node.js uses a package manager to install modules it is very likely that the netmask packaage has been included on nearly every Node.js installation. A package manager is a piece of software that manages what components are loaded in a software framework (operating system, application environment, or runtime). In the case of Node.js developers can use NPM to find and install the components they need in an automatic way. All package managers can handle dependencies which means they will find all the pieces of code (libraries in old school speak) that a package requires. What this means is, if a developer chooses to install package A and package A also requires package B and C, and package B requires D, E, and F, and package C requires G, H, and I, the total list of packages to install are A, B, C, D, E, , G, H, and I. Here’s a real-world example:

This is from my Linux system that shows to install the tcpdump package I need 12 other packages–its dependencies. What this doesn’t show is the dependencies of all those packages but with the package manager for my Linux distribution (Gentoo’s emerge) I can easily show those by adding “emptytree” which means assume nothing is installed and find all packages that need to be installed. Here’s the results:

But that is just the top portion, the list contains a total 477 other packages. This might seem like a lot but tcpdump requires a console which requires an operating system kernel, but if I wanted to run tcpdump my package manager is prepared to install a whole Linux kernel, all the needed system and application libraries to support it, terminal drivers, network infrastructure, and everything else needed by the system just to run a simple console command. This just goes to show how complicated computers have become, and Node.js is no different and that is way it is very likely that netmask has been included in your Node.js runtime.

The problem with NPM’s netmask is that it is used as a very basic component for communicating on a network which is the base medium for which Node.js is designed to facilitate: deliver web content from a web server application. This is like saying that a problem has been found in spark plugs so everything using a spark plug is vulnerable. Imagine the list: cars, lawn mowers, string trimmers, chain saws, airplanes, irrigation pumps, boats, leaf blowers, generators, drones, tanks, etc. The list goes on and on, and that is exactly what the situation with netmask for Node.js. There are approximately 210,000 open source projects that use netmask from NPM. That’s already a huge list and it doesn’t even include proprietary software. As of the week of 3-16 to 3-22 (before this vulnerability was announced) netmask was downloaded over 3 million times.

Tighten those seat belts, this one is going to be big. If you use Node.js you are a target. Get with your developers and make sure they know what is happening. Make fixing this problem their number one priority. An upgrade to version 2.0.0 (though only 11 days old) fixes the problem. Do it now because the pitch is already over and the ball is already speeding to the plate.

October 14 2020

A Singular Key to Security

Mike Stute Resources, Security security

In this series of articles I am going to introduce a method of security that is rarely employed by individuals or companies, but provides great protection from both local and remote exploits. It is extremely good because it renders automated attack code, and even previously written hacking tools, useless. It has been in the arsenal for many years, but since the method used to perform it requires compiling the operating system kernel and applications from source, it has been impossible for closed-source operating systems like Windows, and difficult for open-source, such as Linux. However, newer tools are making it much easier to perform.

To understand how it works, consider the following maze and how it might be possible to program a robot to navigate it.

First a simple command to move ten feet forward which simply turns the wheel motors on for a given number of seconds. This can be called “MF” for move forward. A second command would be called “Turn Right” and would consist of turning the wheel motors on in such a way for a given number of seconds to rotate the robot right ninety degrees.

This can be called “TR” along with a similar one called “TL” for turn left. Now assuming each square is ten feet both in length and width to get the robot through the maze to stand in front of the green wall if it starts on the red dot and faces down would be:

MF MF TL MF TL MF TR MF TR MF MF TL MF MF

We now have a method to allow the robot to navigate all obstacles. Those simple commands get the robot through the maze to stand in front of the green wall. Suppose however, we change a single wall in the maze. It now looks like this

With even that simple change our algorithm to solve it fails. The algorithm not only depends on the maze being the same, but the robot being in the same spot and facing the same direction.

Suppose though we wanted to make sure we could solve the first maze no matter which direction the robot was facing. As long as we get multiple tries with the maze resetting between them, this algorithm works but we need to change our starting conditions.

Instead we do four attempts. First let’s call the current algorithm SOLVE. The we do the following as a main routine to solve the maze

SOLVE
TR SOLVE
TR TR SOLVE TR TR TR SOLVE

Notice that each time we try the solution we turn to right one, two, and three times. This eventually will orient the robot correctly for the solution to work with a maximum of three failed attempts if the robot starts facing left. Hacker tools often try multiple times changing offset values from a known list or simply incrementing through a set of values trying to find the correct one. Each wrong attempt might cause the process to crash, forcing the OS to restart it.

To finish the system we simply need to have a program to execute the commands and the data necessary to drive the routine. We would have to store in the robot’s computer memory the program and the data. It might be something like this table:

Now the computer would simply start running the code stored at row 4 and the robot would begin. At each attempt to solve the maze using SOLVE if it fails, the robot is reset to its original position and another attempt can be made. In this fashion, multiple attempts can be made allowing the SOLVE routine to eventually solve the problem. Of course, computers only use numbers so instead of calling the subroutines by number we will instead call them by number. Location 4 contains the main program which calls all the others. MF is stored at 0, TR at 1, TL at 3, and SOLVE at 3. The data the main routine will use to know which order to call the subroutines is stored at location 5.

Now we have the basis for a simple program to solve the first maze regardless of which direction the robot is facing. The whole system is an analogy for a computer program with the code and data stored in the table. Though very simplified, it represents a program in memory of a computer, and in this case if we think of the maze as an exploit tool to a vulnerability, we have a program that can use that vulnerability to break into a system.

A program calls functions both in external and internal libraries and the locations of these routines can all be calculated from an offset from some known location in the program. It is relative to a given position, perhaps 4,187 bytes forward or 3,209 bytes backwards. If the location of the code is known, data can be changed to break the system in such a way that execution jumps to an unintended location in the original program but instead to the location of the exploit code. There are several special locations for data a program uses including the call stack to store the return addresses in which to jump after execution of subroutines. Such techniques as stack buffer overflows work because this stack is also used to store parameters, variables scoped to the current function (local), and current instance (the this pointer). The call stack has many uses but knowing the exact state of this stack is important for a hacker to the call-return system of return-oriented programming (ROP). Knowing precisely where to place data on this system by causing an overrun is the key to such exploints. Code is simply specialized data to a computer system.

It all becomes very complex since code is local to the current program but also references code in the operating system and external libraries. External library code must be loaded into memory (or if already loaded the address looked up) and these addresses must be put in the program as it is loaded–called a “fixup.” The operating system must maintain a table of these routines in order to reuse them when executing programs. Like programs in memory, specialized files reside on a permanent storage device that contain not only the code for the program, but also all the data required to perform all the fixups to properly modify all external and relative internal addresses a program needs. Since a program can be loaded anywhere in memory (for modern operating systems) many addresses need to use a fixup. This process is called linking and is done dynamically as the operating system prepares the program for execution. The key takeaway is that for many hacking techniques these numbers must be known or found to switch execution from the intended code (the operating system or application code) to the hacker’s code. Very much like the maze, knowing exactly what to do in advance allowed the hacker to create code that could exploit a vulnerability in a system to compromise it. Like the program to solve the maze it is entirely dependent on the maze never changing.

Since all releases of a particular application or operating system are exactly the same, the code that can break one, can break all identical versions. If the above maze is exactly the same, the algorithm will always solve it, but even one change to the maze disables the program. If all the addresses that need fixed up were different for every system the program ran on, a hacker could only design a program to break a single computer. In other words, if each computer had its own maze it would be impossible for a hacker to create a program (at least a reasonably simple one) that could solve the maze. So, can we each have our own maze?

The answer is yes. For open source, this is fairly easy, simply compiling the code with a different version of the compiler is often enough. Reconfiguring the options of the program to exclude features not required for the system (support for hardware not present, network protocols not used, unnecessary features, etc.) or to include features not included in the default. All of these will create different link locations, a different location of the special data areas (call stack, heap), leading to different offsets in executable (for those more technical this means different values in the procedure link table and global offsets table in an ELF file for example). This will make a new maze by building an executable file unique to your system.

If you’re using a source distribution of Linux such as Gentoo, LFS, or my own Toucan Linux Project, you already have a unique kernel, unique libraries, and unique applications. If you’re using a binary distribution such as Ubtunu, RedHat, SUSE, almost all others, or Windows, you’re using the same maze that everyone else is using. Even recompiling the entire distribution will yield the same binaries because all options have already been chosen. You can install kernel source, modify options, and recompile it to at least protect the OS kernel.

But what about software that is closed source? In that case a program specifically designed to read the executable file format, examine the code, and rewrite it to use different offsets, switch the use of registers, examine and randomize the link tables. This is known as a binary rewriter. Many exist including DynamoRIO, Multiverse, Zipr++, RL-bin, and RevARM for Linux on X86-64 and ARM architectures. These can be used for anything from observation of a program operating for performance profiling, but also hardening. They are not easy to use but are fully capable of doing the job, but only in expert hands.For these you’ll need a senior developer, but because it requires specialized knowledge of executable file formats, dynamic linking, and assembly language, it’s a rare developer that can handle the work. These developers work in building developer toolchains and are very hard to come by.

In 2015 a company decided to address this issue and create an easy to use–actually automatic–binary rewriter with the purpose of modifying an executable file to make the program unique in terms of ROP metrics (offsets, registers, jump tables, etc.) explicitly for security hardening. The company is Polyverse and they are busy building a system to polymorph code to make it unique without changing functionality while minimizing performance degradation. Using their service you can build your own maze, one that is unique to your company and can be refreshed every 24 hours. Having your own unique polymorphic applications and operating system is great, but having it change daily is very important as well. Why? We’ll talk about what

mechanisms modern kernels are using to combat ROP attacks and how hackers have responded to bypass these mechanisms. We’ll also talk about what is needed in the future to solve these types of attacks, which are the majority of attacks we find in the wild .

August 27 2020

SD-WAN and Security?

Chris Moffett Resources, Security

We recently asked our security guru if SD-WAN should be considered a part of the security posture in modern networking. Glad we did and below are parts of his response.

Security data sources includes telemetry in the form of logs from applications, firewalls, network devices, and security technologies. Logs are intended to inform an external source what an application or device is doing. But logs are programmed sources of data, meaning a system architect or programmer makes the decision on when to log and what form that log takes. It is a piece of the code that says “log this event or information” that is called as a subroutine through a program. It is entirely possible for a compromise to happen that occurs at a point where an event is not logged, or even that logs can be suppressed as part of the exploit. While logs are a good source of data, especially when collected in mass and correlated together, they represent a secondary level source of information–it can only tell what the software thinks; if the software is wrong, so is the log.

Network traffic is “outside” of an application or even network device. It is traffic on a wire and it must be used to move from one system to another. It is the underlying mechanism of data transport and it is a primary source of information. Regardless of the application state, the network traffic to it is the real data. A remote compromise must generate network data in order to traverse to new system and this network data cannot be faked (though noise can be generated to try to hide it). It is a PRIMARY source data that triggers change of state or events in the destination and that eventually lead to compromise.

All network data has a source and for two-way communication that source must be real not spoofed or faked). This source is indisputable for two systems to communicate (two systems can’t communicate using a spoofed source.) This source can be used to backtrack through a network to determine the device or origin. A log can be made to report the wrong source, but if it is wrong on the wire, it gets delivered to the wrong device or simple tossed as incorrect.

All connections between networks (external, partner, customer, or public) must be watched for 100% attack surface coverage. Why attack a data center where there is generally lots of security, versus the small branch office it connects to where people might be lax and there is much less investment in security?

Finally, SDWAN operates as many devices that are part of the security stack. They operate as router, switch, firewall, security sensor for traffic inspection, point of inspection for viruses, malware, and ransomware (and many others). They are all these devices in a single system instead of deployed in 5 or 6 different devices spread throughout the network and the systems on the network. It is a single point that can not only inspect but control data flow. It is more cost effective because it is a single device. It is easier to deploy and manage because it is a single device. It can be easily upgraded with new functionality because it is a software device that doesn’t require firmware or hardware refresh to gain new functionality.

August 20 2020

More Thoughts to Control Costs in the World of Covid

Chris Moffett Expense Management, Resources

As we close out August, the fifth full month of covid-19 reality, cost pressures on businesses haven’t subsided. Parts of our economy are opening back up and others, like my home state of Texas, reinstated some social restrictions as case rates spiked. The slow decline of those receiving unemployment benefits that began at the end of June reversed recently with right at one million additional workers receiving assistance. It isn’t hard to see how most businesses are still facing challenges to their very survival.

To try and help this is the second article with ideas for fairly easy ways to curtail costs businesses face in their IT infrastructure budget.

*Consolidate Analog Lines*

Analog or POTS services (traditional phone lines) are a must for most companies. They handle alarm systems, fax machines, elevators, and other critical infrastructure components. While needed, they aren’t very flashy and have very little technical impact to an organization’s day to day function. Because of this they are often ignored and their typical low unit cost is treated as a necessary evil. Don’t be fooled, costs for these services add up quickly. One client of ours spends just over $200,000 a year for analog lines at their offices.

All analog lines are not created (or priced) equal. Providers have learned to create packages that sound good but ultimately increase cost. Does an elevator really need call forwarding? Do we need a wiring insurance policy on our fax machine? Doubtful.

A good way to eliminate those fees and only buy what you really need is to use an aggregator to consolidate these services across all locations. Typically, the bulk purchasing these firms bring will lower your bill by 10% but the process we use to help clients onboard with an aggregator also evaluates the configuration of those services and removes unneeded features and functionality that only add to your expense.

*Telecom Audits*

The thought of an audit does not normally stir feelings of joy but in this case it should. Organizations tend to add, remove, change, and shift over the course of business and I’m talking about people and technology. It is inevitable that something will fall through the cracks and more often than not that something costs you money. An audit of technology expenses is a great way to find items that once were important, you still pay for, but no longer fit into the business strategy.

In over 20 years in this industry I have never seen a properly completed audit that did not turn up savings. From the data circuit at a closed office to mobile phones assigned to former employees, the dollars are there.

Before you cringe about another project, partner companies of ours do these audits on a contingency basis and require little involvement from your team. They go through product costs, all the surcharges and other cryptic “fees” added by telecom providers, and when errors are found they claw back as much as the law allows. You have very little to lose and lots to gain.

June 23 2020

What is Agent57?

Mike Stute Resources, Security

Part 1

In 2012 a team in Alberta, Canada introduced the Arcade Learning Environment (ALE) as a method to help develop artificial intelligence. They created an environment that allowed AI agents to play the simple games of the 1970’s on the Atari 2600 VCS using an emulator (Stella), providing methods to input to the emulator from another program, and to provide the output (screen) or internal RAM of the system as input to another program. The paper goes on to demonstrate several methods to use the ALE to train an AI agent to play games and finally proposes these simple games from the early video computer systems as a way to advance AI. In order to fully generate what might be considered specialized AI (an AI that excels at a single task) the paper selected 57 games as a test.

Some of the games are quite easy, such as Space Invaders which requires shooting objects that slowly scroll across the top of the screen while avoiding being shot (just in case you’re young enough to not remember here’s a video). The targets moved back and forth left and right and lowered one space each time they changed direction. If the player failed to hit them all, they would reach the bottom of the screen and the game was over. The other method to lose was the proverbial “running out of lives.” If an enemy missile hit the player three times, the game ended. Occasionally along the top of the screen (requiring the highest skill at deflection shooting), came another object (the mother ship) worth many more points than the individual, but numerous, lower targets. To play only required a left and right movement and pressing the button to fire. Even as simple as it was, the game contained some subtleties.

These subtle aspects of the game required a different play style to maximize the score (the goal – though for me it was hitting more aliens than my father could.) The player could only have a single bullet on the screen at a time and had to wait until it hit an object or disappeared off the top of the screen. A missed shoot meant a long time waiting, a serious penalty for missing. As the player shot the lower targets they started speeding up until the last one moved so fast it could move down one row in the time it took a missed shot to clear the screen. Good players tended to shoot the lower aliens to the point where hitting one more would cause them to perform the first speed advance then shift to hitting only the mother ship until they aliens reached a point low enough to make it dangerous. This produced the highest score with the least amount of danger. But the game had limits, and even though the aliens tended to speed up over the longer game and start lower and lower, they never exceeded a maximum speed or depth and if a player became so good that they rarely died, it was simply faster to clear the lower targets to achieve maximum score in the shortest period of time. However, becoming that good was difficult since at full speed it challenged the human brain to process all the information (retinal neurons to look at the screen), visual processing to decipher it (visual cortex), thinking and prediction (prefrontal synthesis), and move the fingers in time to react (the primary motor cortex stretching over the top of the brain from one ear to ear.) Some players became very good, but death would eventually come though perhaps only from lack of sleep. This game took no reasoning, only simple prediction, and good hand-eye coordination. Other games were more difficult.

One such game was Solaris. In this game the game universe consists of sixteen quadrants each containing 48 sectors that can contain over nineteen different icons to indicate different missions. Choosing a sector can be a free choice or it might be a requirement such as refueling, repairing, or saving a friendly planet. It contains different enemies based on the contents of the sector. Positioned at the bottom, a readout, shows different icons depending on whether the player is in warp or out and it can be damaged requiring the player to use two numbers at right or left to calculate what it normally visually showed. Complicated? Yeah, and there are components I haven’t even touched on like corridors, a completely different visual screen. The goal: navigate all quadrants (each containing 48 sectors) to reach Solaris. This game not only requires great hand-eye coordination, but both strategic and tactical thinking. Add in that Solaris does have a running score counter (the goal is to maximize it) but points are added to the score only at the end of a large goal, making it difficult for an agent to see a reward for a good action.

Another game, Pitfall! works completely differently. Here the world is static, it never changes. Players can memorize what each screen contains and even how to best navigate between them. It consists of screens that look similar but navigating them means quick thinking and fast fingers as they are made up of components that seem randomly placed. To win the player needs to navigate to various screens and collect gold and silver bars using memorization. A timer at the top of the screen counts down from twenty minutes. Being a static game, it might seem a great candidate for an agent, but the screens were diverse enough and the map large enough to require a much larger memory than many agents contained (or complexity of the network.) The problem with Pitfall! is playing it is very, very different than most other titles so an agent trained on Space Invaders and Solaris would find it the new environment completely alien. The 57 games picked create diversity and constitute a challenge for a single agent to learn to play. It is important to understand that this is not a suite of agents designed to learn each game but a single agent with the ability to learn all 57 games and play them above a human level.

Why the Atari 2600? It is simple early device. It only has 128 bytes of RAM, no video buffer (programmers drew the screen using the CPU one line at a time using 4 single line sprite objects) and had only 9 inputs for the player (8 joystick positions and a single button.) This kept games simple (for the most part though some programmers advanced the system far beyond.) I remember the 2600 well and it was rare to read the manual after purchasing a new game. Instead we just started playing, moving the joystick around to see what moved, how it moved, and what the button did. What actions generated score, or other positive rewards and what actions were detrimental. We could intuitively figure out how to play by simply exploring. Some games required exploring and the game state only changed if the player moved. Other games had hidden chambers, secret passages that meant walking into what appeared to be a wall, but in game reality were illusions. Experimentation and exploration, required components for many games, is necessary in any agent.

In 2015, DeepMind released a paper on Double-Q leaning that produced the best agent that could beat the human normalized score for 29 of the 57 games. These were mostly the move and shot at targets while avoiding being shot games (Galaga like) or the move through the maze while avoiding the bad guys games (Pacman like.) In the next version they added short-term memory so an agent could recognize and remember previous actions for on-policy learning (LSTM) and off-policy (DRQN). The next very successful version added an exploration module that allowed the agent to explore using various techniques to try to find how to achieve the goal or change the state of the screen. A good example of this is Montezuma’s Revenge in which the player can stay in the same place for as long as they choose without dying or changing the score. To an agent without exploration, the agent soon learns not to move on this screen as invariably it leads to death. This is because this game’s setting is a cave and many exits are illusionary walls that the player can only find by bumping into every wall to determine if it is real or not. Other screens are completely dark (no light source) but there are still obstacles, structure, and exits though they can only be “felt,” not seen. Finally, DeepMind added episodic memory in order to allow the agent to switch from “learned” mode to “explore” mode when new and different looking screens were discovered. They coupled this with a density model in the exploration engine for the agent to be able to judge how often it had seen a specific element or one like it (familiarity.) This allowed the agent to use the exploration mode to try to change the game state, without this the agent might get stuck doing nothing or a few movements that had no effect, essentially getting “stuck” in a game that required moving from screen to screen like Tutankham, Pitfall!, or Krull.

In the latest incarnation of DeepMind’s Atari 2600 agent they added a meta-controller. Many games require exploration to learn what enemies are valued over others, such as Fishing Derby where the deeper fish are worth more than the shallow fish, and really don’t represent that much more difficulty, only more time. That can be handled in easy games that are static, but not games that have multiple screens like Tutankham or Pitfall! that need to first be fully explored before optimal play can be achieved. Other times it is necessary to balance short-term and long-term goals to optimize game play. Such games as Solaris that only give rewards at the end of a long series of events requires focusing on long-term goals versus games like Astrerix (Taz in the United States) where the only goal is to touch one set of icons while avoiding another. To handle exploration versus learned and short versus long term rewards the concept of a “time horizon” that switches between the various modes to help seek a long term goal versus a short term goal and governs how long to play in learned mode versus how long to stay in explore mode to change the game state. As it turns out, for many games, reaching the highest possible score in the least amount of time means first fully exploring the game, then picking which long-term or short-term strategies lead to optimal play. DeepMind solved this using a type of algorithm from stochastic decision theory called sliding-window upper-confidence bound which is a way to measure the potential of an action by valuing both the previous rewards received but overtime increasing the value of untried actions. In this manner all actions are tried but those that offer no reward or low reward are quickly removed. With this addition to their last agent (Never Give Up – NGU), they created Agent57.

So how did it do? We’ll look at that next week, and dive into the decision on whether Agent57 is a true specialized AI.

June 8 2020

Controlling Costs in the World of Covid

Chris Moffett Expense Management, Resources, Trusted Advisor

We have lived with the new world of quarantine and social distancing for several months and even as parts of our country begin to “open up” many of us in the business world are not certain a return to normal is on the horizon. Most of our clients are evaluating and experimenting how to continue operations with an economy stopped in its tracks. Our team and I have fielded countless calls from those looking for ways to contain expenses while revenues plummet and I thought this was a good time to share a few points that are easy to do and can be very effective to help stem the flow of dollars out the door.

*Remove the toll-free call-in option on your conferencing accounts*

Part of our new normal is dealing with online conference and video calls. Not all companies operate this way but with stay at home orders in place for those non-essential staff, many find their team is spending more and more time using tools like Zoom, GoToMeeting, or WebEx. Unexpected, and certainly, unwanted bills can show up when your staff is still using toll free numbers as they set up meetings. Those 1-800 numbers tend to be the culprit of unwanted bills.

Toll free numbers typically come with an additional monthly fee but more importantly they come with a per minute charge for every person using the call. For example, if 15 people dial into a call for one hour that equals 900 minutes (15 people x 60 minutes) and if the charge is $0.02 a minute, that call costs $18.00. That fee isn’t high but when you multiply it by the number of meetings occurring over the past few weeks the expense can be extremely high. Removing that option from your conferencing platforms only takes a few minutes and I have seen it save $40,000 in 60 days.

*Pay attention to wireless data usage*

Bundling wireless devices such as mobile phones, tablets, or hotspots into a corporate account normally saves a company money when compared to reimbursing employees for usage on their personal devices. Companies buy large data pools that let the tens, hundreds, or even thousands of staff take their small piece of a much larger pie and since all those devices don’t always use the same amount of data, the company saves money by bundling all their staff together. Rarely do all devices start using high amounts of data all at the same time–until a global pandemic.

Wireless carriers have online tools that allow the administrators to see usage and identify which devices (and thus people) are using a lot. In times like these it probably isn’t employee abuse, it’s just business as we now know it. However, that doesn’t mean the company is stuck. Options such as increasing the data pool, working with staff to find a better internet option, slowing down the automatic updates from some mobile device management tools can all save the data and reduce potential overage bills each month. Another option many of our clients use is a third party management tool that tracks usage and alerts you before a device crosses an expensive threshold. A client of ours recently experienced wireless expenses 6 times higher than previous months and most of that growth was preventable.

*Work with your providers*

Most office data networks are seeing greatly reduced traffic right now meaning those expensive Internet or MPLS connections are not providing much return. Many network operators have deferment policies or programs that will let you turn them down temporarily without a penalty. Specifically with our clients, we have helped them reduce circuit usage by an aggregate of roughly $250,000 just in the month of April. In most cases, you can turn them down just by asking. In other cases, you may be able to trade a few extra months on the end of a long contract for short term relief that reduces the drain on cashflow. Reaching out and asking the question doesn’t hurt and let’s face it, you will always miss 100% of the shots you never take.

We do believe business will come back and the entrepreneurial spirit will find ways to overcome these troubling times but it never hurts to save some money now. Some of these actions will carry forward and help the bottom line for months or years to come. Hopefully at least one of these items is helpful and stay tuned in the next few days. I’ll have some other actions a company can take to maximize cash over the near term and into the future.

Photo by John Guccione www.advergroup.com from Pexels

May 7 2020

CIO Perspectives: What is a Trusted Advisor and why do I need one?

Jon Baird Resources, Trusted Advisor CIO, Trusted Advisor

For over 30 years of my career operating in various roles in IT, the one constant I faced at every level of the organization was the expectation placed on me to deliver solutions. As an analyst on a QA team, I was expected to challenge a piece of software and expose its defects. As a developer, the solutions came in the form of (hopefully) minimal defect, highly functional and performant pieces of code. As a development manager, those pieces had to come together in the form of a software system to solve for a key element of a business transaction. And as a CIO, coupling those systems and its data with infrastructure to solve for complex business challenges proved the pinnacle of solutioning.

My recent transition away from traditional IT as a consumer of technology and services into a role as a provider of technology solutions has proven to be an eye opening experience in several ways. First, I had to truly embrace the value a Trusted Advisor can bring to the critical discipline of technology solutioning. Second, I had to realign my way of thinking in how I consumed technology in the past to current consumption models. And third, and most importantly, I had to open my eyes to the vast array of emerging technology solutions that are dramatically changing the landscape of capabilities and that, quite frankly, I was completely unaware of in my role as a CIO. I have discovered numerous technologies delivered by countless providers that I had never even heard of and solutions that I likely would have considered highly valuable to my IT strategy had I had the awareness.

As a CIO, my key objectives, driven by the business, focused my line of sight directly on my business partners, my team, my budget, and my current capabilities. And when those objectives required capabilities outside those I currently possessed, I looked to technology providers for solutions. But how did I start this journey? Well, maybe someone on my team had knowledge of a specific solution set that would solve the problem but more than likely, they were too laser-focused on daily tactical issues and firefighting to have had the bandwidth to explore the wide array of available solutions. Maybe I’d consult the latest Gartner Magic Quadrant and gather together a list of the 30 vendors that might have a solution but how do I begin to manage that list? Maybe I’d reach out to my VAR although their line-card is limited and by definition, non-agnostic. Or maybe I’d drop large sums of money on consultants.

What I truly need to face this challenge is someone I can trust. Someone who brings subject matter expertise to the problem and can quickly consume high-level goals and objectives and whittle down seemingly limitless providers into a manageable solution set. Someone who is not just an order taker but can sit on my side of the table and partner with me in the discovery process. Someone who can leverage their industry expertise to help negotiate the best deal. Someone who is vendor-agnostic and thoroughly understands the technology landscape. Someone whose success is tied to mine and who has an incentive to build a long-term relationship. And someone whose compensation is aligned with delivering a fully functioning solution. Ultimately, someone who has my back. I need a Trusted Advisor.

A good Trusted Advisor is defined by several key characteristics. First and foremost, my Trusted Advisor needs to take a “what’s in it for you” viewpoint. They need to exhibit broad subject matter expertise. They should act as a confidant and be someone with whom I can discuss the business and IT-aligned strategy but also be a tactical thinker.

The need to be outcome oriented. They need to be a true partner and someone for whom I wish to have a seat at the table. They should be willing to question and disagree with a proposed direction but provide encouragement and support when the decision is made. They need to be an avid listener and thoughtful questioner. Ultimately, they need to be an extension of me and my team.

So, to my fellow CIOs (and CTOs and CISOs alike), whether you’re fortunate enough to find yourself in an organization with the breadth to stay on top of the broad set of technologies available to you (which very few of us are) or you have limited resources and don’t have the luxury of an awareness of the solution landscape because you and your team’s focus is on driving business outcomes, seek out a partnership with a Trusted Advisor. And when you do, make sure they can bring the value you know to expect and when that happens, you will have found yourself a new ally in the daily fight to grow your business through technology.

April 21 2020

What is a Trusted Advisor?

Keith Hatley Trusted Advisor

For the better part of my 27-year career, I’ve been a sales leader working for different service providers in technology. I have had the opportunity to work for some really incredible companies and been a part of organizations that were considered best in class. In spite of that, however, the products and services I represented often didn’t solve the problems my prospects were trying to solve – we couldn’t meet the budget requirements, the implementation timelines, the billing needs, etc…or, in many cases, we just weren’t the right fit.

I was always trying to understand the business impact of the problems the client was looking to solve to see if we could fill a need…and, if we couldn’t, we wanted to lose fast. Don’t waste time. Ask questions. Qualify. Disqualify. When I sit back and think about it, I’m actually reminded of the game of baseball in that I would lose more than I would win. Derek Jeter said baseball is “a game of failure. If you fail seven out of 10 times people think you’re great.” Sales is the same way…or at least it was.

You see there’s a movement in technology right now and the procurement model is changing. Rather than selling “a” service there is a new breed of solution providers called Trusted Advisors. A Trusted Advisor is an overall solutions provider that doesn’t represent a particular product but represents an entire industry of services. They act as an extension of your team, they focus on delivering business outcomes, and they help you navigate the confusion in the market caused by the accelerating pace of change in IT. They work side by side with your team to design, procure, implement, and support even the most complex IT solutions. Quite simply, a Trusted Advisor is the best friend a CIO will ever have.

There’s a confluence of many factors that are allowing this to happen:

Technology Providers are moving away from dedicated sales and support; leveraging sales agents/partners to reduce costs.

Enterprise Clients out-tasking services to allow focus on what they do best…leaving commodity functions to other, more experienced partners; leaning on those partners for advisement, strategy, and support.

Services Evolution with new services (Cloud, Zero-Trust, IoT, SD-WAN), clients looking at multiple (new) suppliers, with an emphasis on “solutions” and/or as-a-service. Pace of change happening so fast that businesses are needing to rely on partners.

These factors are also forcing an evolution in the “partner” space which, in turn, provides the competition benefits to today’s businesses.

Enterprise clients need to recognize the shift and realize the value these Trusted Advisors can bring. They free up the Enterprise to focus on their business and innovation while the Trusted Advisor focuses on the return of high dollar spend. These Trusted Advisors engage in technology audits, solution research, provider selection, and ongoing account management and support.

For me, there’s no longer a question whether “my” product or service fills the gap. There’s no longer a need to “lose fast” so I can quickly move on to find the 1 of 3 (or more) prospects that were a good fit. Now, I get to listen. I get to really listen and learn and I have the opportunity to bat a “thousand”. I get to sit on the same side of the desk as my client to work side by side and help deliver technology solutions to solve business problems and that’s an incredible feeling.

Cloud Communications Group is a technology expense management and IT advisory firm and acts as Trusted Advisors to our clients. We are revolutionizing the discipline of IT procurement by saving our clients time and money and leveraging some of the world’s most sophisticated as-a-Service platforms for IT decision making in driving key business outcomes.