Our own, Mateusz Szostok has two sessions at KubeCon around K8s Service CatalogRead more

Mateusz Szostok, Piotr Kopczynski and Pawel Kosiec have several talks at San Diego Cloud Native MeetupRead more

Governance

Overview

This section defines the ownership policy and the decision-making process within the Kyma organization.

Kyma working model

Scope

All repositories in the Kyma organization must follow the official guidelines, contributing rules, and the governance process to assure quality and consistency.

The Kyma project also includes the Kyma Incubator organization. It is a place where all new projects start in a more relaxed environment that facilitates their rapid growth. At that stage, they do not have to comply with all rules that govern the Kyma organization. Once the incubating project is ready to become a part of the main Kyma organization, adjust it to all standards.

Add a new repository to Kyma's organizations

Create a GitHub issue in the community repository and describe the purpose of the new repository, proposed name and explanation which organization it should go to, kyma-project or kyma-incubator.

Ownership policy

Kyma repositories are owned by code owners who are a group of people with special privileges in the repositories of the Kyma organization. Each repository has a separate CODEOWNERS file located at its root. The file specifies persons who have the ability to approve contribution to the part of the repository they own, after a detailed review of the related pull requests (PRs). Although the name suggests only the code ownership, the CODEOWNERS file is not only about the code but the content in general. Apart from the developers, you can define any relevant parties as code owners. For example, technical writers are set up as the owners of all .md documents in the Kyma repositories and SIG/WG members are the owners of their SIG/WG's folders' content.

Code owners (aka maintainers) responsibilities

With great power comes great responsibility. Maintainers not only review and approve PRs but also truly care about their projects.

Every code owner is expected to:

  • Contribute high-quality code and content
  • Communicate and collaborate with other code owners to improve the ownership process
  • Perform a thorough review of incoming PRs and make sure he/she follows the contributing rules
  • Approve only those PRs in which the contributor made the requested improvements
  • Check if the related CI tests have completed successfully before approving the PR
  • Make sure that the PR approval flow runs smoothly
  • Proactively fix bugs
  • Perform maintenance tasks for his/her projects
  • As an assignee, treat issues assigned to a current release with priority
  • Monitor Slack
  • Triage GitHub issues and perform pull request reviews for other maintainers and the community. Follow the labels workflow document
  • In general, continue to be willing to spend at least 50% of his/her time working on Kyma (~2.5 business days per week).

Process for becoming a maintainer (aka codeowner)

  • Express interest to the existing maintainers that you or your organization are interested in becoming a maintainer. Depending on whether you are an individual or a corporate contributor/maintainer, a different contributor license agreement applies.
  • Express what the areas of Kyma that you want to engage with are. Kyma, because of its goal, is complex, and it is impossible to become a maintainer in every area. Make sure you get support from a proper group of maintainers.
  • Becoming a maintainer generally means that you are going to be spending a substantial part of your time (>50%) on Kyma for the foreseeable future. Among other responsibilities, this means:
    • creating and merging pull requests
    • reviewing pull requests
    • creating or commenting on issues
    • helping on Slack and other community communication channels.
  • We will expect you to start contributing increasingly complicated PRs, under the guidance of the existing maintainers.
  • We may ask you to do some PRs from our backlog.
  • As you gain experience with the code base and our standards, we will ask you to do code reviews for incoming PRs (i.e., all maintainers are expected to shoulder a proportional share of community reviews).
  • After a period of approximately 3 months of working together and making sure we see eye to eye, a candidate for maintainer should create an issue in the community repository with a list of his/her current contributions. The existing maintainers will confer and decide whether to grant the maintainer status or not. While we make no guarantees on the length of time this will take, we expect it to be no longer than 3 months.

NOTE: To be accepted as a maintainer and be added to Kyma GitHub organizations, you need to have two-factor authentication enabled on your GitHub account.

When does a maintainer lose the maintainer status?

  • If a maintainer is no longer interested or cannot perform the maintainer duties listed above, he/she should volunteer to be transitioned to emeritus status.
  • If a maintainer is unresponsive for at least 3 months, he/she can be removed from the maintainers list immediately.
  • In extreme cases, this can also occur by a vote of the maintainers per the voting process below.

How to suggest a change in maintainers list?

To suggest a change in the ownership of a given repository part, create a PR with the required changes in the CODEOWNERS file in the project's repository. The required number of code owners needs to approve the PR for the changes to take place. Read here on how to set up and modify owners of the given repository folders and files.

The name of the removed maintainer should be added to the emeritus file with short info about what areas were covered by this person.

Conflict resolution and voting among maintainers

In general, we prefer technical issues and the maintainer membership to be amicably worked out between the persons involved. If a dispute cannot be decided independently, the maintainers can be called in to decide an issue. If the maintainers themselves cannot decide an issue, the issue will be resolved by voting. The voting process is a simple majority in which each maintainer receives one vote.

For complex cases when there is a situation that we need a decision that affects more than just one group of maintainers, the Decision Making process applies.

Decision-making

In general, the Special Interest Groups (SIGs) and Working Groups (WGs) make decisions that affect the project, including its structure, functionalities, components, or work of the project teams. However, the organizational decisions and those that relate to the product strategy are made by the Kyma Steering Committee.

SIGs and WGs follow the lazy consensus approach to decision-making which assumes that:

  • All SIG/WG members have an equal voice in the decision-making process.
  • Silence is consent. By default, lack of objections to a proposed decision means a silent approval.
  • Any objections are good opportunities for healthy and constructive discussions.

NOTE: The described approach only concerns the decisions made by SIGs and WGs. It does not affect any Kyma decisions made during daily team activities.

To see the current state of processed decisions in the Kyma project, go to the project's decision board.

The SIG/WG decison-making process is as follows:

Decision-making process

Create a decision record

Create an issue and choose the Decision record template. Provide the information as requested in the template and set the issue label to decision. The issue can contain all necessary information, reference a document, or an open pull request with a more detailed proposal. Additionally, set the corresponding WG or SIG label such as sig/core.

NOTE: In some cases, the decision label is set for an existing issue during the triage, which indicates the need for a decision. In this case, use the Decision record template and copy it to the issue description accordingly.

Reach a consensus

Take all necessary actions to reach a consensus no later than the decision due date:

  • Send a link to the Decision record issue to the related SIG or WG mailing list (For Core SIG the easiest way to do it is to add a new topic in google group as then it is automatically emailed to the mailing list) and post it in the relevant Slack channels. If people relevant for the decision are not part of the mailing list, add them explicitly to the email communication. Communicate the decision to be made clearly to the groups affected by it and invite them to check the proposal.

  • Clarify and discuss the decision content and the proposal as needed. Use the mailing list, relevant Slack channels, the related pull request, or comment directly on the Decision record issue.

  • Feel free to communicate the decision proposal during the upcoming SIG or WG meeting and ask its members for feedback. Encourage the discussion and bring up any objections early in the process.

  • Those who created the proposal work with those who had objections to either prepare an improved solution or decline the proposal.

NOTE: Discussions lead to changes in the decision record or the proposal, or end up with no changes required. If someone suggests a substantially different approach, ask its supporters to write a counter proposal and to submit it in a separate pull request.

Close the decision

Once you reach the consensus:

  • Add the status change in the Decision record issue with either Accepted or Declined.
  • Close and merge the pull request with the accepted proposal. Make sure that the merge comment contains one of the keywords to automatically close the Decision record issue. Otherwise, close the Decision record issue manually.

NOTE: If there are any pull requests with counter proposals to the decision record, close the related pull requests with rejected proposals without merging. State the reasons for the rejection in the closing comments.

Lack of consensus

Engage the Core SIG leaders if there are still unresolved objections by the decision due date. They will work with the Kyma Steering Committee if necessary to reach the final decision as soon as possible and close the issue.

Revisit a decision

Raise an explicit request to revisit a decision or to review it. To request a revision:

  • Create a new decision record in the respective repository and populate the Affected decisions parameter.
  • Specify the Due date parameter and set a revision date that is not earlier than the revision date of the original decision.
  • Explain in the Context and Consequences sections why you propose another approach and a new decision.

The SIG or WG addresses the request for revision after the original decision record reaches its revision date. Exceptionally, you can suggest to revisit a decision earlier if the ultimate decision makers support and request it. They can trigger the process if they identify a major positive impact on the project, substantial improvement, or the community interest.

Kyma Steering Committee

The purpose of the Kyma Steering Committee is to:

  • Maintain the mission, vision, values, and scope of the project
  • Engage in the resolution of conflicts that cannot be solved by other governance rules
  • With the help of the community, refine the governance model as needed
  • Agree to the new SIG or WG proposals or raise concerns about them

The size of the committee

To make sure the committee can always make a majority decision, we have three members. The ultimate goal is to have five members (at least two individual ones) and never four.

Decision-making

Whenever the committee is involved and needs to make a decision, the decision is always made by a majority of vote. In the case of three members, it is 2/3, and in the case of five it is 3/5.

Members

This is a GitHub team that lists all current members of the Kyma Steering Committee:

We are aware that at the moment Kyma is steered by a single company, SAP. Our goal is to see on the list of members only one SAP representative and four representatives from outside of SAP. We commit that once there are five members on the list and we have another 2 candidates ready to step in, two SAP representatives will step down to achieve truly open governance of the project with five representatives form different parties.

How to become a member of the committee?

The committee is open for any contributors, individual or corporate. These are the rules we apply:

  • In case of individual contributors, you need to be an active Kyma maintainer for at least 2 major components.
  • A single company can have only one employee in the committee. It is up to the company to suggest a candidate that will acts as a representative of active maintainers for the given company. The representative, however, does not have to be an active maintainer himself/herself.
  • In the case one company has one representative in the committee already, and another member starts working for this company, one of the members must resign once there are other candidates to replace him/her.

Once you fulfill one of the requirements, create a GitHub issue in the community repository and describe:

  • Why you want to join the committee
  • Who you represent
  • What your current contribution to Kyma was
  • What components you officially maintain and provide links to all the activities that you took as a maintainer

The members of the committee openly make comments under the issue. The comment should include information on what the vote is and the reasoning for the vote. The candidate will receive clear feedback in the case the request is rejected, with advice on what must be done to improve the request and be accepted.

Issues workflow

In the Kyma project, we use GitHub Issues for tracking development process, and ZenHub to manage the issues on a team and sprint level and to have a clear overview of the work across all Kyma repositories.

This document explains:

  • How the issues and pull requests workflows are organized in the Kyma project
  • How issues triage is organized
  • Which tools are used on every stage of the workflow

Used labels

Our statement is to:

  • Use default labels provided by GitHub
  • Introduce new labels only if necessary

Default labels

The default labels provided by GitHub are as follows:

Custom labels

Here are the custom labels introduced by the Kyma team. The labels colors are provided in brackets in Hex code:

  • WIP (#ECF44F) indicates that an issue is not ready for further processing.
  • decision (#ED635E) indicates that an issue is related to a decision.
  • priority/critical (#FB0104) indicates the top-priority of a given issue.
  • area/{CAPABILITY_NAME} (#3CB913) indicates which capabilities are related to a given issue. You can assign more than one area label an issue.
  • area/quality (#3CB913) indicates that an issue is related to a quality topic.
  • area/performance (#3CB913) indicates that an issue is related to a performance topic.
  • security/{SEVERITY} (#2D51F9) indicates a security issue based on its CVSSv3 severity, either low, medium, high, or critical.
  • sig/{SIG_NAME} (#E99694) indicates which Special interest group (SIG) identified the issue and is responsible for further follow-up on the issue.
  • wg/{WG_NAME} (#E99694) indicates which Working group (WG) identified the issue and is responsible for further follow-up on the issue.

Issues triage

Here is the flow diagram explaining how issues triage is performed:

There are eight different stages of the triage:

StageDescriptionLabels
ValidityAssess the validity of the issue (whether it is taken for the further triage and proper classification).invalid, duplicate, wontfix, question
KindDifferentiate whether the related issue is a new feature or a bug.enhancement, bug, test-failing
DecisionCheck if the issue is related to a decision.decision
HelpIdentify issues that do not have high priority and can be taken by the community.help wanted, good first issue
SecuritySpecify the CVSSv3 severity with the support of the security team.security/{SEVERITY}
SIG/WGClarify which SIG or WG is involved in this issue and is responsible for the further follow-up on the issue.sig/{SIG_NAME}, wg/{WG_NAME}
PriorityPrioritize issues in the general Kyma backlog to select those which are the most critical and should be taken as first.priority/critical
AreaClarify which capabilities or qualities are involved in a given issue.area/{CAPABILITY_NAME}, area/quality or area/performance

Backlog

The Kyma backlog contains issues that went through the triage, are not closed, and have labels added (except for the issues with the question label). Backlog prioritization is realized by assigning issues to Kyma milestones (ZenHub Release) and assigning the priority/critical label. Critical issues assigned to the current milestone have the highest priority.

NOTE: Issues are taken from the main Kyma backlog by different teams that are responsible for specific areas of Kyma. This is the actual workflow in Kyma which allows distinguishing which team works on a specific issue. This approach allows easy work in team sprints using ZenHub board. We are aware that our teams' names may be cryptic for the external community, but at the moment we are not able to provide any better solution. If you have a better idea, your feedback would be highly appreciated. Sorry for the inconvenience.

Team sprints

Team Sprint is modeled as a GitHub milestone named with the following pattern: Sprint_{TEAM_NAME}_{NUMBER}. During the planning, a team selects issues from the backlog considering:

  • priority (high priority first)
  • area (default capability of the team first)
  • dependencies (unblock others)

Contributors move the issue to the In Progress column on the ZenHub board when they start working on it. When the work is done, the issue is closed.

Team backlog (optional)

If any team wants to keep team backlog (assign issues they want to work on later), they can assign it to the Github milestone with no due date named with following pattern: Backlog_{TEAM_NAME}.

Stale issues

To keep the Kyma backlog clean, the bot monitors all repositories in the organization. It marks old, inactive issues with the stale (Hex: #E4E669) label and closes them after a given period of time. For configuration details, check this sample file.

Although the bot helps us to keep the backlog clean, we regularly monitor its activities to make sure it is not closing issues that are still valid and important for Kyma. The Kyma team reviews this ZenHub board and acts on them as follows:

  • Closed issues:
    • If the issue is still valid, reopen it and remove the stale label from it.
    • If the issue is invalid, change the stale label to a more relevant one and add a comment that provides background and explains why the issue remains closed.
  • Open issues:
    • If the issue is valid, remove the stale label from it.