Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

flutter.engine: init #212328

Merged
merged 2 commits into from
Jun 15, 2024
Merged

flutter.engine: init #212328

merged 2 commits into from
Jun 15, 2024

Conversation

RossComputerGuy
Copy link
Member

@RossComputerGuy RossComputerGuy commented Jan 23, 2023 •
edited
Loading

Description of changes

Fixes #201574

This PR is mostly working, only thing to resolve now is a ninja issue. However, I believe this PR is just about ready for review and this PR has everything needed to make building the Flutter engine possible.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.05 Release Notes (or backporting 22.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@RossComputerGuy RossComputerGuy force-pushed the flutter-engine/init branch 3 times, most recently from 8857160 to 55cd4f9 Compare January 23, 2023 23:28
@ofborg ofborg bot added 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 10.rebuild-linux: 1 labels Jan 23, 2023
@RossComputerGuy RossComputerGuy force-pushed the flutter-engine/init branch 3 times, most recently from 0512ec6 to 6bfd43d Compare January 24, 2023 00:59
@ofborg ofborg bot added 11.by: package-maintainer This PR was created by the maintainer of the package it changes and removed 10.rebuild-linux: 1 labels Jan 24, 2023
@RossComputerGuy
Copy link
Member Author

Flutter engine now starts ninja!

@RossComputerGuy
Copy link
Member Author 

flutter-engine-jit_release> Done. Made 953 targets from 324 files in 520ms
flutter-engine-jit_release> building
flutter-engine-jit_release> Building flatc
flutter-engine-jit_release> [88/88] LINK ./flatc_party/libcxx/libcxx.stamp.o_rust.o[KooK[K[KKK
flutter-engine-jit_release> Building blobcat
flutter-engine-jit_release> [565/565] LINK ./blobcat/impeller/blobcat/blobcat_lib.stampder.oter.o[K[Kble.o[Kfee25/src/third_party/icu/common/icudtl.dat icudtl.dat
flutter-engine-jit_release> Building gen_snapshot
flutter-engine-jit_release> [514/514] LINK ./gen_snapshott/runtime/libdart_precompiler.ailer.stampKger.ook.oer_x86.o[Klper.o
flutter-engine-jit_release> Building impellerc
flutter-engine-jit_release> [1415/1415] LINK ./impellercmpeller/compiler/compiler_lib.stampmpmp_val.validate_builtins.oes.oK.os.o.oinux:clang_x64)ang_x64)
flutter-engine-jit_release> Building Flutter Engine library
flutter-engine-jit_release> [113/1313] ACTION //flutter/impeller/entity:gen_blob_embed_gles_entity_shaders(//build/toolchain/linux:clang_x64)
flutter-engine-jit_release> FAILED: gen/flutter/impeller/entity/gles/entity_shaders_gles.h gen/flutter/impeller/entity/gles/entity_shaders_gles.c
flutter-engine-jit_release> python3 ../../../../../../../build/flutter-engine-src-857bd6b74c5eb56151bfafe91e7fa6a82b6fee25/src/flutter/impeller/tools/xxd.py --symbol-name entity_shaders_gles --output-header /nix/store/k987lqbgl09glaf3xfrckbb8ybpahjsh-flutter-engine-jit_release-857bd6b/lib/flutter/out/jit_release/gen/flutter/impeller/entity/gles/entity_shaders_gles.h --output-source /nix/store/k987lqbgl09glaf3xfrckbb8ybpahjsh-flutter-engine-jit_release-857bd6b/lib/flutter/out/jit_release/gen/flutter/impeller/entity/gles/entity_shaders_gles.c --source /nix/store/k987lqbgl09glaf3xfrckbb8ybpahjsh-flutter-engine-jit_release-857bd6b/lib/flutter/out/jit_release/gen/flutter/impeller/entity/genlib_gles_entity_shaders.shaderblob
flutter-engine-jit_release> Traceback (most recent call last):
flutter-engine-jit_release>   File "/nix/store/k987lqbgl09glaf3xfrckbb8ybpahjsh-flutter-engine-jit_release-857bd6b/lib/flutter/jit_release/../../../../../../../build/flutter-engine-src-857bd6b74c5eb56151bfafe91e7fa6a82b6fee25/src/flutter/impeller/tools/xxd.py", line 95, in <module>
flutter-engine-jit_release>     main()
flutter-engine-jit_release>   File "/nix/store/k987lqbgl09glaf3xfrckbb8ybpahjsh-flutter-engine-jit_release-857bd6b/lib/flutter/jit_release/../../../../../../../build/flutter-engine-src-857bd6b74c5eb56151bfafe91e7fa6a82b6fee25/src/flutter/impeller/tools/xxd.py", line 51, in main
flutter-engine-jit_release>     assert os.path.exists(args.source)
flutter-engine-jit_release> AssertionError
flutter-engine-jit_release> [124/1313] CXX obj/flutter/impeller/renderer/backend/gles/gles.pipeline_library_gles.o
flutter-engine-jit_release> ninja: build stopped: subcommand failed.

Mostly working, we just need to fix this issue now. I checked the assert and it looks like renaming the stuff in configurePhase breaks stuff. I'll fix that and we should have the engine working.

@RossComputerGuy
Copy link
Member Author

Commits have been reordered and squashed, it's now tidy.

@RossComputerGuy
Copy link
Member Author

As requested by @FlafyDev, the engine is now opt-in as the source code is 21GB to fetch (and Flutter's tool requires the source code to be available to use the engine) and a single engine build can be around 4GB to 7GB.

@RossComputerGuy RossComputerGuy force-pushed the flutter-engine/init branch 3 times, most recently from e975f8c to 1730fa0 Compare June 12, 2024 02:32
@ofborg ofborg bot added 10.rebuild-darwin: 11-100 and removed 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin labels Jun 12, 2024
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/1741

@RossComputerGuy RossComputerGuy force-pushed the flutter-engine/init branch 2 times, most recently from 0e295c2 to b76bbec Compare June 13, 2024 22:47
@RossComputerGuy
Copy link
Member Author

Update script now supports the engine.

tomberek
tomberek reviewed Jun 15, 2024
View reviewed changes
pkgs/development/compilers/flutter/engine/package.nix
];

postBuild = ''
ln -s /nix $out/nix
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ln -s /nix $out/nix
mkdir -p $(dirname $out${builtins.storeDir})
ln -s ${builtins.storeDir} $out${builtins.storeDir}

or?

Suggested change
ln -s /nix $out/nix
ln -s ${buildEnv ....} $out/some-path

not sure what direction to go here

@RossComputerGuy RossComputerGuy force-pushed the flutter-engine/init branch 3 times, most recently from f433b78 to ed691c3 Compare June 15, 2024 03:05
@tomberek
Copy link
Contributor

Result of nixpkgs-review pr 212328 run on x86_64-linux 1

1 package failed to build:
  • flutterPackages-source.v3_22 (flutterPackages-source.stable)
44 packages built:
  • expidus.calculator
  • expidus.calculator.debug
  • expidus.calculator.pubcache
  • expidus.file-manager
  • expidus.file-manager.debug
  • expidus.file-manager.pubcache
  • firmware-updater
  • firmware-updater.debug
  • firmware-updater.pubcache
  • flet-client-flutter
  • flet-client-flutter.debug
  • flet-client-flutter.pubcache
  • fluffychat
  • fluffychat-web
  • fluffychat-web.pubcache
  • fluffychat.debug
  • fluffychat.pubcache
  • flutter (flutter322 ,flutterPackages-bin.stable ,flutterPackages-bin.v3_22 ,flutterPackages.stable ,flutterPackages.v3_22)
  • flutter313 (flutterPackages-bin.v3_13 ,flutterPackages.v3_13)
  • flutter316 (flutterPackages-bin.v3_16 ,flutterPackages.v3_16)
  • flutter319 (flutterPackages-bin.v3_19 ,flutterPackages.v3_19)
  • flutter323 (flutterPackages-bin.beta ,flutterPackages-bin.v3_23 ,flutterPackages.beta ,flutterPackages.v3_23)
  • flutterPackages-source.v3_19
  • hover
  • intiface-central
  • intiface-central.debug
  • intiface-central.pubcache
  • localsend
  • localsend.debug
  • localsend.pubcache
  • python311Packages.flet
  • python311Packages.flet-runtime
  • python311Packages.flet-runtime.dist
  • python311Packages.flet.dist
  • python312Packages.flet
  • python312Packages.flet-runtime
  • python312Packages.flet-runtime.dist
  • python312Packages.flet.dist
  • rustdesk-flutter
  • rustdesk-flutter.debug
  • rustdesk-flutter.pubcache
  • yubioath-flutter
  • yubioath-flutter.debug
  • yubioath-flutter.pubcache

@RossComputerGuy
Copy link
Member Author

@tomberek What's the build failure for that 1 package? Is it the pushd error? If it is then I already fixed that and pushed it.

@tomberek
Copy link
Contributor

Result of nixpkgs-review pr 212328 run on x86_64-linux 1

45 packages built:
  • expidus.calculator
  • expidus.calculator.debug
  • expidus.calculator.pubcache
  • expidus.file-manager
  • expidus.file-manager.debug
  • expidus.file-manager.pubcache
  • firmware-updater
  • firmware-updater.debug
  • firmware-updater.pubcache
  • flet-client-flutter
  • flet-client-flutter.debug
  • flet-client-flutter.pubcache
  • fluffychat
  • fluffychat-web
  • fluffychat-web.pubcache
  • fluffychat.debug
  • fluffychat.pubcache
  • flutter (flutter322 ,flutterPackages-bin.stable ,flutterPackages-bin.v3_22 ,flutterPackages.stable ,flutterPackages.v3_22)
  • flutter313 (flutterPackages-bin.v3_13 ,flutterPackages.v3_13)
  • flutter316 (flutterPackages-bin.v3_16 ,flutterPackages.v3_16)
  • flutter319 (flutterPackages-bin.v3_19 ,flutterPackages.v3_19)
  • flutter323 (flutterPackages-bin.beta ,flutterPackages-bin.v3_23 ,flutterPackages.beta ,flutterPackages.v3_23)
  • flutterPackages-source.v3_22 (flutterPackages-source.stable)
  • flutterPackages-source.v3_19
  • hover
  • intiface-central
  • intiface-central.debug
  • intiface-central.pubcache
  • localsend
  • localsend.debug
  • localsend.pubcache
  • python311Packages.flet
  • python311Packages.flet-runtime
  • python311Packages.flet-runtime.dist
  • python311Packages.flet.dist
  • python312Packages.flet
  • python312Packages.flet-runtime
  • python312Packages.flet-runtime.dist
  • python312Packages.flet.dist
  • rustdesk-flutter
  • rustdesk-flutter.debug
  • rustdesk-flutter.pubcache
  • yubioath-flutter
  • yubioath-flutter.debug
  • yubioath-flutter.pubcache

tomberek
tomberek approved these changes Jun 15, 2024
View reviewed changes
Copy link
Contributor

@tomberek tomberek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a large closure and large build. Watch out for issues with Hydra.

@tomberek
Copy link
Contributor

tomberek commented Jun 15, 2024 •
edited
Loading

I'm not a fan of the

ln -s /nix $out/nix

It hardcodes "/nix/", which i'd like to avoid (perhaps with builtins.storeDir?) and perhaps ensure it is "/nix/store" instead.

I'm merging as-is to make progress, but please come back to this.

@tomberek tomberek dismissed stale reviews from eclairevoyant and drupol June 15, 2024 22:53

Recommendations were adopted.

@tomberek tomberek merged commit f7aae7b into NixOS:master Jun 15, 2024
20 checks passed
@RossComputerGuy RossComputerGuy deleted the flutter-engine/init branch June 15, 2024 22:54
@RossComputerGuy
Copy link
Member Author

m not a fan of the

ln -s /nix $out/nix

Yeah, I didn't quite get to it. I'll take care of that this evening.

@RossComputerGuy RossComputerGuy mentioned this pull request Jun 16, 2024
Merged
18 tasks
@gepbird gepbird mentioned this pull request Oct 22, 2024
Closed
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tomberek tomberek tomberek approved these changes

@milahu milahu Awaiting requested review from milahu

@FlafyDev FlafyDev Awaiting requested review from FlafyDev

@eclairevoyant eclairevoyant Awaiting requested review from eclairevoyant

@ericdallo ericdallo Awaiting requested review from ericdallo

@babariviere babariviere Awaiting requested review from babariviere

@hacker1024 hacker1024 Awaiting requested review from hacker1024

@mkg20001 mkg20001 Awaiting requested review from mkg20001

@SuperSandro2000 SuperSandro2000 Awaiting requested review from SuperSandro2000

@gilice gilice Awaiting requested review from gilice

@drupol drupol Awaiting requested review from drupol

Assignees
No one assigned
Labels
8.has: clean-up 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 11.by: package-maintainer This PR was created by the maintainer of the package it changes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

flutter: Packaging the Flutter engine