I see that you survived my earlier posts. Thanks for sticking with me. Now, government financial management is a dry topic, so I’ll try to keep this short and to the point. Depending on the type of contract issued by the program office to the contractor, different types of financial management requirements come into play.
The most complex and difficult contracts often rely on a contractor submitting earned value management (EVMS) reports to the government as the way of providing transparency into the costs of the projects. The basis of EVMS is the Work Package and, unfortunately, contractors can manipulate the definition of and type of work packages to hide or obscure financial issues during contract performance. I once had a large DoD contractor tell me that I was not allowed to see what work packages they were using on my contract, so there was no way for me to understand, much less validate that their financial performance was satisfactory. I should not have to accept a “trust me” card from any government contractor when it comes to ensuring that our taxpayers money is spent appropriately.
Here is where applications like Jama Connect can make a difference and drive that transparency. A work package is a set of work to satisfy specific requirements. I am not going to bore you with defining EVMS, but I will tell you that EVMS includes multiple cost metrics which are used to calculate the Schedule Performance Index and Cost Performance Index, which measure the contractors’ current state of meeting the requirements on time and on budget.
Jama Connect could be used to track the contractor’s work packages and where the contractor will update the financial metrics. This would provide the program office insight into how the work packages were developed and their validity.
If a program office was really ambitious, it could require the contractor to relate each work package to the government approved requirements, such that it would be now possible to identify what requirements are on the cost and schedule critical paths, introducing overruns and delays, and hopefully allow the contractor and government to put in mitigation activities. This would allow for an honest tradeoff between requirements, cost, and schedule based on actual data instead of intuition.
Jama Connect allows the contractor to document work packages and to relate the work packages to requirements, deliverables, and any other items in Jama Connect to provide context and decision quality information.
The benefit of implementing such relationships may not be worth it. It really depends on what is the best way to decompose the requirements and create a logical set of work packages. A forced alignment of those two things may just not make sense, but then again, if they do, then it opens up a lot of value to both the contractor and program office leadership. Tools can’t replace the insight of an experienced program manager, but most program offices have only a few program managers with extensive experience.
The benefit of incorporating financial management with the requirements within Jama Connect is because they are always related. Having a single place with the information needed to make a decision is critical in supporting the right decision to be made. As decisions are required, Jama Connect supports doing those trade-offs and scoping any required contract modifications. It also provides a platform to document and record these decisions.
Welcome to the second post of this series – if you haven’t already, go back and read the introduction to the government program offices series to learn more about how Jama Software supports government program offices and more about me and my qualifications.
Today, we’ll be exploring the art and science of contract requirement development within federal government program offices and how Jama Connect could be used. While tailored for federal civilian and DoD program offices, many of the lessons learned here will directly apply to corporate program offices as well.
With a few exceptions, the federal government does not build anything. They negotiate contracts with civilian contractors to build things. The program office defines the contractual requirements, conducts a source selection to award the contract to the best contractor, and then manages the contractor’s performance to ensure that the requirements were met. Or at least, that is the extremely simplified version of the Federal Acquisition Regulation (FAR) process.
Today, let’s look at the two key program office requirements documents used to solicit proposals from industry, the Request for Information (RFI) and the Request for Proposal (RFP). Sometimes these documents go by different names, but I’ll stick to RFI and RFP.
The RFI is the first public requirements document used to determine if there is sufficient interest in industry such that a competitive acquisition could be done, and to solicit expert input on the initial set of requirements so the government doesn’t overlook anything that would cause problems in the acquisition later.
The RFP is the more formal requirements document used to solicit formal proposals from industry, and is the basis of the contract. It is primarily filled with boilerplate legal and contracting language, with a few tailored sections which describe the actual requirements and how the proposals will be evaluated.
The goal of the RFI is to gather information and insights from industry so you don’t screw up the RFP. The quality of the answers provided in response to an RFI is proportional to the quality of the questions asked in the RFI. The RFI is also a strong signal to industry on the seriousness and professionalism of the program office issuing the RFI, and will influence their decision to submit a well thought out proposal or not.
In general, the program office will have a set of user requirements, either generated internally to the program office or externally by the organizations the program office is supporting. It is easy to just publish an RFI with those requirements and ask the industry “can you meet these?” – but that does not provide the program office much insight. The best RFI’s are well thought out, and are asking well crafted questions to industry that not only cover the user requirements, but provide insight into gaps in the requirements, potential acquisition pitfalls that should be avoided, and to expose a range of options for how to optimize the acquisition.
So what is Jama Connect’s role in this? Jama Connect is the platform where the questions in the RFI are built, collaborated on, and optimized. For example, below is a screenshot of a sample Jama Connect project where we have the user requirements and a set of potential RFI questions. By using Jama Connect, it is now possible to effectively collaborate with stakeholders throughout the government. Jama Connect also can help organize all of the responses such that the actual lessons are learned.
The beauty of Jama Connect is that it enables continuity between the RFI and RFP. Getting input from industry is useless if it is unable to be processed and used to optimize the requirements in the RFP. The RFP is where requirements “get real,” and the set of requirements in the RFP will evolve into the formal contractual requirements. Both the program office and contractor benefit from having a well defined, objective, and easily understood requirements.
Now that industry has provided responses to the RFI, the program office, it is up to people like me to sift through all of the comments and determine what changes need to be made to the requirements prior to publishing the RFP. This is where Jama Connect shines. Jama Connect allows me to effectively evaluate each input from industry and then update the RFP accordingly. As the RFP gets closer to completion, the more scrutiny it undergoes through the wide range of government stakeholders including the program office, user representatives, test organization, legal office, contracting office, just to name a few. Jama Connect allows me to keep a single source of truth throughout this complicated review process and allows all of the stakeholders to have detailed visibility into the requirements as needed.
Now is a good time to explain why Jama Connect makes a difference and why doing this in a word processor or spreadsheet just isn’t good enough. Jama Connect allows for a holistic development of the contract requirements while ensuring that only the publicly available information is released to the contracting community. This allows the program office to put everything into Jama Connect and to export only what is needed to publish the RFI/RFP, instead of spreading out this information in different documents, email traffic, memos for record, and so on.
Let’s look at one specific program office requirement, and the context that be included in Jama Connect. Remember, this isn’t just a repository of requirements, this is the place to build, collaborate, and approve requirements. In Jama Connect you write it, share it, refine it, approve it, and finally export it to the right format.
So, since I’m a former Air Force, we’ll pick on one of my favorite requirements of all time. The formally approved requirement was “The system shall provide decision quality information.” Now, as you might imagine, context is everything and thus the requirement that must be in the RFI/RFP to be useful is going to reflect the context of that higher-level requirement.
As you see, the original requirement created a lot of discussion, and was eventually replaced with more specific requirements. Not only that, we now have a way to include context related to source selection evaluation, risk, costs, and testing along with the contract. Now, this requirement in Jama Connect can be collaborated on within the government program office and other stakeholders. This means that everything is in one place, and if there is a change to the requirement, it is immediately understood as to where else that change may impact. Perhaps as important, it provides program office leadership insight into the program office’s actual understanding of the requirements and their preparation to go through a source selection.
Finally, the RFP is submitted to the acquisition authority for approval. Once approved, the contracting office will publish the RFP and the fun of source selection begins.
Stay tuned for next post in this blog series, publishing on Thursday, June 10th.
With President Biden signing the new Executive Order on Improving the Nations’ Cybersecurity. I thought it would be worthwhile explaining how Jama Connect can be used by both Cloud Service Providers (CSPs) and Third Party Assessment Organizations (3PAO) to accelerate and automate development of FedRAMP compliant and authorized offerings.
For those unfamiliar with FedRAMP, it is the US Government’s cybersecurity framework for commercial IaaS, PaaS, and SaaS providers which, if successful, allows those CSPs to deliver cloud based services to federal agencies. Historically, all federal applications had to be hosted within federal controlled data centers, and FedRAMP enables delivery from CSP controlled commercial infrastructures. Each of the major cloud infrastructure providers have FedRAMP approved environments, and thus, PaaS or SaaS companies can leverage similar environments to their existing commercial applications for FedRAMP.
The cornerstone of FedRAMP is the System Security Plan (SSP). The SSP is the documentation package to basically describes how the CSP has developed the system in compliance with the required security controls, and how the CSP will operate the system in a compliant manner with the requirements. The SSP is interesting because it simultaneously documents the requirements and the technical/operational design of the system.
Depending on the complexity of the system, the SSP package may be 400-1500 pages when printed. This is not something that you print out and expect people to read. However, this is what the 3PAO and the FedRAMP Sponsor (the federal agency taking you through the process) read in detail to determine compliance and to understand the risks associated with accepting the system as developed, or if additional security measures are required.
One of the huge challenges with FedRAMP is to get today’s DevOps teams to not only read the requirements, but to also design/implement the system in accordance with the requirements and to participate in the necessary documentation of how they built the system. Having tried to just shove a very large Word document at them in the past, I’ll testify that approach is one to guarantee failure. In reality you’ll have a small team with someone like me on it, coordinating FedRAMP requirements decomposition, writing documentation, and closely working with the DevOps team to ensure that they 1) understand the requirements, and 2) the documentation accurately represents the implementation.
If you have made it this far in my blog post, I hope you have a sense of the complexities and the requirements/documentation challenges with FedRAMP. But since this blog is on Jama Software’s website, I should pivot slightly and explain how Jama Connect can make FedRAMP much easier. While there are several tools available that will help you write an SSP, most of those tools are not fully developed requirements management tools, and as such do not allow for flexible approaches, customization, requirements decomposition and tracking, flexible workflow, and so on. I have found it better to adapt Jama Connect to work with FedRAMP, than to take other tools and try to make them into requirements management tools. It is a tradeoff, and Jama Connect does not currently generate an SSP at the push of a button, but that isn’t as important as getting the requirements right the first time and getting into the market and generating revenue.
So, what can Jama Connect do for you if you are getting into FedRAMP?
Jama Connect can be your reference library so you can avoid hunting for documents on the internet
Jama Connect can provide valuable guidance on the best practices regarding FedRAMP development
Jama Connect can be used to actually develop the SSP, and to then create and track the required product feature requirements or tasks to ensure that system is built and operated in a compliant manner
Jama Connect can be used to share the SSP artifacts with the 3PAO and to get their feedback
Jama Connect can be used to identify and collect the evidence required for the audit
Jama Connect can be used to identify and collect the evidence required for continuous monitoring
I’ve created a very barebones sample Jama Connect project to explain some of these capabilities in more detail. Most of the effort in building this was uploading the NIST controls from a spreadsheet and building out the organization based on my experiences.
Before jumping into screenshots and so on, it has also been my experience that not everyone on the FedRAMP team, especially those team members in DevOps will want to use Jama Connect. While it may be impossible to fully collaborate with them without having them at least occasionally go into Jama Connect, there are two Jama Connect features worth addressing. The first is that it can integrate with multiple existing DevOps tools, so tasks in Jama Connect can show up as tickets in Jira and etc. This allows DevOps to continue to operate in the tools they are used to, yet the overall FedRAMP effort to be managed and documented in Jama Connect. The second feature is that if I make a comment in Jama Connect and I tag another user, then Jama Connect can send that user an email. That user can then respond by email. This is two ways that non-core team members can effectively participate with the team without needing to be Jama Connect experts.
Now, for this demo project, I’ve assumed that the entire team is able and willing to work within Jama Connect. Hopefully this will help describe the scope of work necessary for FedRAMP and how Jama Connect can be that one central working space to manage the required FedRAMP requirements and documentation.
Let’s start by looking at the Jama Connect dashboard for my sample project.
The ability to actually track the status of every piece of required documentation and each of the security controls is why you never want to just build an SSP package in Word or Google Docs. At one glance I can see where we are in getting the SSP completed, and how we are doing with each control. I can also see everything that is assigned specifically to me, so I can click on any one of those links and start editing.
The SSP package is complex and if you do it right, you’ll have multiple authors, reviewers, and so on. If I was the sole person responsible for building the entire package then I wouldn’t need Jama Connect and my company would fail. FedRAMP is a team sport, and the dashboard is your game plan.
The beauty of this is that unlike other tools that allow you build the documentation, Jama Connect can be configured to track anything. So, if there are specific product feature requests that must be met to be compliant, that too can be included in the dashboard. The same goes for all of those DevOps tickets that are used to facilitate building the actual system.
Let’s take a closer look at the SSP itself.
This is what an SSP looks like in Jama Connect, or at least the outline of an SSP. Now, the new cybersecurity Executive Order goes into detail on the requirements for improved secure development practices, so let’s jump into SA-11, Developer Security Testing and Evaluation.
For the demo project, I’ve included the NIST and FedRAMP guidance for each control with the response. Now, if we scroll down a bit.
Below the guidance, are the fields that the CSP must fill out and be audited against. This is where someone like me would start writing. However, if I do not know how we do testing, I can use Jama Connect to start asking questions.
I can ask Josh for details on this. He can answer by logging into Jama Connect or simply by responding to the email. Jama Connect will track unanswered questions so you can blast out questions and not get lost in the responses. Try doing that in Google Docs or email. Josh can also just click on the link in the email and then see the requirements or any of the documentation stored in Jama Connect.
Unfortunately Josh responds with “Ummm we don’t do this,” so now we need to create a DevOps user-story to get them to install the appropriate static code analysis tools. Easily done in Jama and now we have both the control, and a related user-story.
Now, I can track the completion of the user stories or tasks that must be accomplished to either make what is written in the SSP true.
So, Jama Connect can be used to build your SSP package and to assign user-stories/tasks as required across your company to either complete the SSP, or bring what is said in the SSP into reality. If it can do this, and your teams can effectively use Jama Connect for this, it has already paid for itself (at least in my opinion).
When you get the SSP complete and your Sponsor approves the System Assessment Plan, it is time to go into audit. For the audit, the 3PAO will provide you a list of all of the evidence required, which generally is either documentation, interviews, or a demonstration. The CSP then has to provide the documentation, schedule the interviews and conduct the demonstrations. This can also be facilitated by Jama Connect. Some 3PAO’s have their own online tools for this, and some will just send you a spreadsheet. The good news is that you pay them, and if you’d like them to submit and track the audit completion in Jama Connect, that may be doable depending on the 3PAO.
Above I have just a simple evidence request on background checks. It is easy enough to give the 3PAO access to Jama and for them to upload their evidence requirements. Then the CSP can attach the evidence to the requirements and the 3PAO can review and approve the evidence submission. This is easy to do in Jama through component level permissions (you don’t want the 3PAO going through everything in Jama) and workflow. This can then be added to the dashboard so you have a near real time assessment of how the audit is going.
The last aspect of Jama Connect I want to share with you is continuous monitoring, and the Plan of Actions and Milestones (POA&M) specifically. After you survive the audit and get your authorization, you enter into continuous monitoring. One important part of that is establishing and maintaining the POA&M, which is where the CSP documents the risks associated with the system and the plan associated with mitigating the risks. The POA&M is provided to the government monthly. While there are other tools designed specifically for POA&M management, again those are niche IT spends and aren’t suited for requirements decomposition and task assignment.
Above is one simple POA&M item. Normally a CSP may be monitoring between 50 and 200 POA&M items at one time. Just like the controls, Jama Connect allows you to fully document the POA&M and then to create related user-stories or tasks, and then track those to completion. It also allows you to start tracking items that may not yet be on the POA&M, but that if are not remediated within the required timeframe will be put on the POA&M. Again, just like everything else with Jama Connect, how the POA&M field types are configured and the workflow is all customizable, so it can be adapted to what works best within the CSP.
In summary, Jama Connect can be a critical tool if used correctly for any CSP desiring to get into the FedRAMP business. Jama Connect’s flexibility, collaboration capabilities, and ability to integrate with all existing tasking tools can significantly streamline SSP development, audit preparation, and support the successful completion of the audit. The effectiveness of Jama Connect in FedRAMP is completely dependent on the CSP’s willingness to leverage it, and there is nothing about Jama Connect from a capabilities standpoint that restricts it’s effectiveness. While it may not offer the one-button SSP generation solution that niche SSP tools may offer, the benefits of Jama Connect significantly outweigh the few hours of effort to build the SSP in Word for the PMO.
And while the perfect solution for this does not exist, Jama Connect does break the requirements and development out of an unreadable Word document and into the light, where the entire CSP can focus their efforts to achieve FedRAMP authorization in the most efficient manner possible.
Stay tuned for John Allion’s 8-part series on how Jama Connect supports government program offices and read the series introduction here.
In this blog series, I’m going to discuss how Jama Connect is a potential game changer for US federal and Department of Defense government program offices. I spent 24 years in the Air Force with 17 of those years in one or another government program office, or as the Air Force likes to call them System Program Offices, or SPOs. Every three to four years, I’d move to a new base and join a new SPO.
My role within the Air Force was that of a “developmental engineer” which is unique to the Air Force. None of the other Services have a direct counterpart to that role, and the closest comparison I’ve come up with in industry is what happens when you mix a solution architect, enterprise architect, and technical project manager together. My job was to take user generated requirements and then to deliver systems that meet those requirements.
Practically all of my career in the Air Force was in dealing with requirements. Sometimes it was the Federal Acquisition Regulations, sometimes it was a Statement of Work (SOW), and other times it was the actual technical requirements for one weapons system or another. After I retired from the Air Force I joined a very large enterprise IT company and that is where I first encountered Jama Software. It didn’t take long after being exposed to Jama Connect that I realized how this cloud-based requirements management system could have improved my life in the Air Force, if we just had Jama Connect and used it effectively.
After annoying people at Jama Software that I now count as my friends for years, they have given me the opportunity to share my views with you. To put this in perspective, I am not a Jama Software employee, not paid in anyway by them, and yet I’m investing a significant amount of my time to write these blog posts. I do this because I want to make life in any government program office to be better for those today than it was for me. I don’t want it to take a full year to come up to speed for a newly assigned acquisition professional. I don’t want people to flat out forget critical requirements resulting in the waste of tens of millions of taxpayer dollars.
As the first step in any 12-step program, the government needs to acknowledge that they have a problem with requirements. I think one of the best written summaries, at least for the DoD, was done by MITRE, and is called Modernizing DoD Requirements. While they do not promote any specific technical solution or platform to support their recommendations, it is easy to understand how Jama Connect would be critical to streamline how those recommendations become reality.
MITRE also released a paper in 2017 that explored the DoD acquisition workforce which identified that over 50% of the workforce would be eligible for retirement in the next 10 years (now 6 years), and that it objectively took a long time to become an expert in this field. This is where technology can make a difference. Any tool, including Jama Connect, that can shorten that learning curve is critical in maintaining the proficiency of the acquisition workforce.
While MITRE focused these two reports on DoD program offices, I could pull similar findings for multiple federal government program offices as well.
I hope I’ve justified why I’m writing this series sufficiently, and I hope you enjoy or at least find useful my take on how Jama Connect can help government program offices improve their core mission of delivering cost effective solutions that meet validated user requirements. I’m breaking this series into six different posts, each addressing a pain point I’ve had to live through and how Jama Connect can improve the processes or outcome of those processes.
As you read this, I do want to highlight that while I’ve offered some screenshots to highlight the requirement management concepts being discussed, the project I created for this was extremely simple. Please don’t read too much into how the Jama Connect project is organized and take this what it is, just an overly simplistic example. Jama Connect is extremely customizable (sometimes to its own detriment), and can be tailored to the item types and workflow required by any specific Program Office.
In this blog series, I’m going to discuss how Jama Connect is a potential game changer for US federal and Department of Defense government program offices. I spent 24 years in the Air Force with 17 of those years in one or another government program office, or as the Air Force likes to call them System Program Offices, or SPOs. Every three to four years, I’d move to a new base and join a new SPO.