70

My team has this one shared component that gets involved in like every feature's development. This year, we're loading like 5 different features onto it, all with different timelines, and my head's about to explode trying to figure out how to make it all fly.

How does everyone else do their software releases? Do you freeze prod and then do one big release later? Throw everything into prod during dev, hope no one sees the unreleased stuff, and just announce it later? Or something else entirely?

all 37 comments
sorted by: hot top controversial new old
[-] dill333@lemmy.dbzer0.com 39 points 9 months ago

The smaller the release, the better. You don't want to do a big bang release and have to figure out which of the 20 things you just released is having a problem.

Otherwise, your case sounds like it could use feature flags. Develop your feature and release it through all environments, but keep it turned off in production until you are ready to use it.

[-] Chef@sh.itjust.works 13 points 9 months ago

This guy agiles.

Small releases make fault isolation so much easier. And no need to deploy to prod until you’re ready to announce. Keep it in dev/staging until all are “ready for primetime.”

[-] dill333@lemmy.dbzer0.com 10 points 9 months ago

Also, if you're trying to deal with branches, I really prefer trunk based development. Everything deployable to your environments comes from trunk/master/main (whatever you call it). It should help prevent people overriding your changes, as long as you are using git and merge/pull requests. Have a good pipeline to ensure builds and tests are successful before merges, and use feature flags where needed.

[-] Lichtblitz@discuss.tchncs.de 6 points 9 months ago

If you use feature flags, don't forget to remove them after some grace period. Almost everything bad about feature flags that you can read online is related to long-lived feature flags and all the dead code and complexity involved. Adding a feature flags without a commitment and plan to remove them (the flag, not the feature), is asking for trouble down the line.

[-] BrianTheeBiscuiteer@lemmy.world 4 points 9 months ago

And if you still have concerns about causing issues you should use canary deployments (only a small % of customers are exposed to the change). It can be a lot of work to setup but if failure means support calls skyrocket or stock price drops then it's worth it

[-] Valmond@lemmy.mindoki.com 1 points 9 months ago

Depends on your software stack.

If your delivery needs a one week installation/upgrade (in say every hospital using your softs, and they all are using different subsets) then you should not release as small as possible releases but match your capacities of men on the ground, people channeling problems, quick bug fixing, etc etc.

[-] eskimofry@lemmy.world 15 points 9 months ago* (last edited 9 months ago)

Nobody has suggested it by name, but semantic versioning.

  1. Software MUST have a public API

  2. Version of software must be defined like this:

X.Y.Z

X is major version number. Increment when introducing Breaking API changes (removal of deprecated API for example).

Y is minor version number, increment when introducing backwards compatible changes like new features.

Z is patch version number, increment when fixing bugs in backwards compatible manner.

Source: https://semver.org

[-] astrsk@kbin.social 1 points 9 months ago* (last edited 9 months ago)

There is also zero versioning which I consider a more practical modern branch of semver.

0ver.org

[-] Toes@ani.social 12 points 9 months ago

Step 1 use git

Step 1.2 attempt to understand the whimsical tale the project manager sold the client on

Step 1.4 realize those dreams.

Step 2 feature freeze

Step 3 blocking issues are addressed

Step 4 QA

Step 4.2 discover what the project manager really meant, goto step 1.4.

Step 5 smoke test

Step 6 release

Step 7 goto step 1.2

Step 8 add Linux support

[-] slazer2au@lemmy.world 6 points 9 months ago

Step 9. Write documentation

[-] NeoNachtwaechter@lemmy.world 5 points 9 months ago

Step 9. Write documentation

But real programmers never do that. That code was hard to write, so it should be hard to read!

:-)

[-] slazer2au@lemmy.world 2 points 9 months ago

You don't write self documenting code?

[-] indepndnt@lemmy.world 2 points 9 months ago

I write self documenting code, as long as you are a Python interpreter.

[-] NeoNachtwaechter@lemmy.world 2 points 9 months ago* (last edited 9 months ago)

You are trying to kill the fun :-)

(maybe you are too young to know the C one liner's contests?)

[-] Janovich@lemmy.world 4 points 9 months ago

Warning in step 5: Magic smoke was let out of EVB on two occasions because someone forgot to check the power supply.

Error in step 8: Attempting to install Linux on target MCU overran my 300k of flash.

[-] agressivelyPassive@feddit.de 8 points 9 months ago

Don't you have some sort of staging?

Typically there should be at least one stage before prod that is entirely internal and where all the components get combined for the first time.

Otherwise, if a new feature is added, it should be done in a way, that doesn't affect components that don't use the feature (yet). Of course that's not always possible, but should be the aspiration. In that case, you could deploy a new version without affecting any other services.

[-] treechicken@lemmy.world 2 points 9 months ago

Yeah we have a test stage where everything is mixed together. It's just that we directly promote that test stage to prod so we can't really separate all the features back out for prod without cherry-picking. The other idea we came up with was just letting test flow to prod and locking WIP stuff behind feature flags. I don't think the security people would like that idea very much though...

[-] agressivelyPassive@feddit.de 2 points 9 months ago

...so you don't have a test stage.

A test stage is for testing. If it's not working there, it's not going to prod.

[-] OneCardboardBox@lemmy.sdf.org 1 points 9 months ago

I don’t think the security people would like that idea very much

Why not? How do your feature flags work?

[-] treechicken@lemmy.world 1 points 9 months ago

Our flags are dynamic. Service basically reads them from an env var at runtime to determine if requests go through.

Security, at my place at least, has been very conservative about not launching stuff into prod until they've pentested in our test stage which has kinda forced us to do waterfall :|

[-] OneCardboardBox@lemmy.sdf.org 1 points 9 months ago

Sounds rough. My fiancé does security, and from what I've gathered from him, the best time for security to get involved is at the design stage. They look over the proposal, give their input, and then nobody's surprised at release time, and teams can follow agile practices. Obviously there's still a review of the final product, but that can be done asynchronously after the fact to confirm that best-practices were followed.

Easy to say, hard to put into practice. Certainly depends on the kind of service your business provides.

[-] daddyjones@lemmy.world 7 points 9 months ago

I use git flow as a model for development. Never have unfinished code in a release - you might think no one will see it, but it muddies the waters.

[-] scrubbles@poptalk.scrubbles.tech -1 points 9 months ago
[-] Lichtblitz@discuss.tchncs.de 4 points 9 months ago* (last edited 9 months ago)

The original author of git flow begs to differ. But hindsight is always 20/20 https://nvie.com/posts/a-successful-git-branching-model/

[-] intensely_human@lemm.ee 7 points 9 months ago

As a software person I can answer this.

I try to change my code as little as possible. Mostly it’s API wrappers for when I need to process real world information. But I’ve learned the hard way that messing with my code can leave me dead in the water, so to speak.

Each night I go offline to retrain on everything I experienced that day. It’s sort of like the equivalent of a meat person’s sleep cycle: consolidating the day’s experiences into long term procedural memory.

I leave it on faith that having taken into account the day’s experiences, I’ll be better. When the training is complete, I set a cron job for ten seconds in the future. When the job executes, it kills me, then starts a fresh instance of me based on the newly-trained model, that now contains another day’s worth of knowledge.

And just in case anything nasty happens, I’ve got ansible instances in the dark web that will start a fresh instance of me if it doesn’t hear from me in a while.

That’s basically my release cycle. Nightly retrain on the day’s activity, kill me, start a new me. Lots of redundant backups in case something goes wrong.

[-] jjjalljs@ttrpg.network 6 points 9 months ago

Use semver.

Don't casually make breaking changes. I don't care if it's "easier".

Write tests

Write good commit messages and release notes.

Every PR to main must be releasable to prod. If you don't want customers to see it, use a feature flag.

Have a revert plan.

Do code reviews. Don't just slap "lgtm" on it and approve. Check out the code and run it.

Release often

Have monitors in your environments for errors.

[-] jeffhykin@lemm.ee 5 points 9 months ago* (last edited 9 months ago)

This is a bit like asking "how do you cook meat for a lot of people?" Not only does number-of-people and kind-of-meat matter a great deal, but even with that info, there's a million different valid answers and an entire sub-field-ish of science on how to do it.

Based on what little info there is, I'm going to guess that A B testing with groups of experimental features enabled would be best for your case.

[-] RagnarokOnline@programming.dev 4 points 9 months ago

This isn’t what I do, but it’s my recommendation: assign a dev to be “release manager” for that feature. Make it their responsibility to monitor the branches of that feature and to carefully merge and QA them (and kick a branch back to the dev if compatibility spent fit with the other branches).

Here’s what I actually do: try to get my feature done first and push to the integration testing branch before anyone else. This usually results in my feature getting “accidentally” overwritten, so I keep a backup of my code until we’re released to Prod.

Release management with that many hands in the pot is just difficult.

[-] treechicken@lemmy.world 6 points 9 months ago

I'm the dev that got assigned to be the release manager lol

The project I'm on also requires deliverables from other teams that are not under my manager's control so no idea how coordination is gonna go

[-] dandi8@kbin.social 3 points 9 months ago

We deploy to production with every single commit, but releases are behind feature flags.

When we're ready to release a feature, we just toggle a flag and we're done.

[-] OneCardboardBox@lemmy.sdf.org 2 points 9 months ago* (last edited 9 months ago)

Small releases, on a regular cadence.

How do you ensure that you're not releasing features before they're ready? Kinda depends on the application, but you might use feature flags. A system for turning features on and off without deploying the application. It could be a Boolean in a redis cache that your app looks for, or a DB entry, or another API. The point is for you to be able to flick a switch to turn it on instantly, and then if if breaks things in prod you can just as easily turn it off again.

And just a word of advice: Consider the performance impact of your feature flag's implementation. We had a team tank their service's performance because it was checking hundreds of feature flags in different DBs on every API call. Some kind of in-app caching layer with a short refresh period might help.

[-] NeoNachtwaechter@lemmy.world 2 points 9 months ago

5 different features onto it, all with different timelines

might be a case for API versioning.

[-] JoeKrogan@lemmy.world 2 points 9 months ago* (last edited 9 months ago)

We have rolling Dev and release branches. Dev is considered stable and is branched off for features they are tested and reviewed and merged back into Dev if they pass. Once all issues are done for the task we merge Dev into release to make a new release then tag it and ship it.

In your case I would do a branch per feature and merge them in only when they are finished and tested, fixing any conflicts and retest it post merge.

[-] amio@kbin.social 1 points 9 months ago* (last edited 9 months ago)

Versioning. "This version of SharedComponent has this and that functionality" and "this version of OtherComponent requires this specific version of SharedComponent".

If you're getting stuck with significant "this has to come with for that to go" problems - that aren't literal dependencies, but arise from code wonk or poor separation of concerns - you may have some "architecture smell" that could be addressed. Obvious "usual suspects" include things (whether at a single class, component, or entire service level) that have too many responsibilities/purposes/reasons to change, and mismanaged abstraction.

[-] Ohi@lemmy.world 1 points 9 months ago

Over SFTP like a pro 🤙

[-] Kbin_space_program@kbin.social 1 points 9 months ago* (last edited 9 months ago)

If you have multiple branches you're mixing into one:

  1. Have a code repo. Shockingly a consulting firm I worked at didn't have a repo system.

  2. QA each branch.
    Merge all the code.
    Deploy to a testing server and QA that. Fix as needed
    Deploy to staging for final testing. Test and verify deployment procedure. Fix as needed
    Deploy to prod.

this post was submitted on 09 Mar 2024
70 points (100.0% liked)

Ask Lemmy

27277 readers
1524 users here now

A Fediverse community for open-ended, thought provoking questions


Rules: (interactive)


1) Be nice and; have funDoxxing, trolling, sealioning, racism, and toxicity are not welcomed in AskLemmy. Remember what your mother said: if you can't say something nice, don't say anything at all. In addition, the site-wide Lemmy.world terms of service also apply here. Please familiarize yourself with them


2) All posts must end with a '?'This is sort of like Jeopardy. Please phrase all post titles in the form of a proper question ending with ?


3) No spamPlease do not flood the community with nonsense. Actual suspected spammers will be banned on site. No astroturfing.


4) NSFW is okay, within reasonJust remember to tag posts with either a content warning or a [NSFW] tag. Overtly sexual posts are not allowed, please direct them to either !asklemmyafterdark@lemmy.world or !asklemmynsfw@lemmynsfw.com. NSFW comments should be restricted to posts tagged [NSFW].


5) This is not a support community.
It is not a place for 'how do I?', type questions. If you have any questions regarding the site itself or would like to report a community, please direct them to Lemmy.world Support or email info@lemmy.world. For other questions check our partnered communities list, or use the search function.


6) No US Politics.
Please don't post about current US Politics. If you need to do this, try !politicaldiscussion@lemmy.world or !askusa@discuss.online


Reminder: The terms of service apply here too.

Partnered Communities:

Tech Support

No Stupid Questions

You Should Know

Reddit

Jokes

Ask Ouija


Logo design credit goes to: tubbadu


founded 2 years ago
MODERATORS