Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize the OptionKey class more #14248

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 51 additions & 10 deletions mesonbuild/options.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
from __future__ import annotations
from collections import OrderedDict
from itertools import chain
from functools import total_ordering
from functools import lru_cache
import argparse
import dataclasses
import typing as T
Expand Down Expand Up @@ -98,7 +98,8 @@ class ArgparseKWs(TypedDict, total=False):
'vsenv',
}

@total_ordering
_OPTIONS_CACHE: T.Dict[int, OptionKey] = {}

class OptionKey:

"""Represents an option key in the various option dictionaries.
Expand All @@ -108,23 +109,26 @@ class OptionKey:
internally easier to reason about and produce.
"""

__slots__ = ['name', 'subproject', 'machine', '_hash']
__slots__ = ['name', 'subproject', 'machine', '_hash', '_as_tuple']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm really not sure about this one. Tuple representation is only used for comparison, and I doubt we really need to sort OptionKeys. Especially since we now use the hash for equality comparison.


name: str
subproject: str
machine: MachineChoice
_hash: int
_as_tuple: T.Tuple[str, MachineChoice, str]

def __init__(self, name: str, subproject: str = '',
machine: MachineChoice = MachineChoice.HOST):
machine: MachineChoice = MachineChoice.HOST,
hash_: T.Optional[int] = None):
# the _type option to the constructor is kinda private. We want to be
# able to save the state and avoid the lookup function when
# pickling/unpickling, but we need to be able to calculate it when
# constructing a new OptionKey
object.__setattr__(self, 'name', name)
object.__setattr__(self, 'subproject', subproject)
object.__setattr__(self, 'machine', machine)
object.__setattr__(self, '_hash', hash((name, subproject, machine)))
object.__setattr__(self, '_hash', hash_ if hash_ is not None else hash((name, subproject, machine)))
object.__setattr__(self, '_as_tuple', (self.subproject, self.machine, self.name))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This starts to look very very very similar to a named tuple 🙂


def __setattr__(self, key: str, value: T.Any) -> None:
raise AttributeError('OptionKey instances do not support mutation.')
Expand All @@ -150,17 +154,34 @@ def __setstate__(self, state: T.Dict[str, T.Any]) -> None:
def __hash__(self) -> int:
return self._hash

def _to_tuple(self) -> T.Tuple[str, MachineChoice, str]:
return (self.subproject, self.machine, self.name)
def __lt__(self, other: object) -> bool:
if isinstance(other, OptionKey):
return self._as_tuple < other._as_tuple
return NotImplemented

def __le__(self, other: object) -> bool:
if isinstance(other, OptionKey):
return self._as_tuple <= other._as_tuple
return NotImplemented

def __eq__(self, other: object) -> bool:
if isinstance(other, OptionKey):
return self._to_tuple() == other._to_tuple()
return self._as_tuple == other._as_tuple
return NotImplemented

def __lt__(self, other: object) -> bool:
def __ne__(self, other: object) -> bool:
if isinstance(other, OptionKey):
return self._as_tuple != other._as_tuple
return NotImplemented

def __ge__(self, other: object) -> bool:
if isinstance(other, OptionKey):
return self._to_tuple() < other._to_tuple()
return self._as_tuple >= other._as_tuple
return NotImplemented

def __gt__(self, other: object) -> bool:
if isinstance(other, OptionKey):
return self._as_tuple > other._as_tuple
return NotImplemented

def __str__(self) -> str:
Expand All @@ -175,6 +196,26 @@ def __repr__(self) -> str:
return f'OptionKey({self.name!r}, {self.subproject!r}, {self.machine!r})'

@classmethod
def factory(cls, name: str, subproject: str = '', machine: MachineChoice = MachineChoice.HOST) -> OptionKey:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't tried to implement it, but I think you can obtain the same effect without forcing the users to call the .factory() class method by implementing __new__() instead.

"""A cached initializer for OptionKeys

:param name: The name of the option
:param subproject: the subproject the option belongs to, defaults to ''
:param machine: the machine the subproject belongs to, defaults to MachineChoice.HOST
:return: An OptionKey
"""
# This doesn't use lru_cache so that we can re-use the calculated hash
# in a cache miss situation.
_hash = hash((name, subproject, machine))
try:
return _OPTIONS_CACHE[_hash]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is unlikely, but two objects can have the same hash but not be equal. I'm not sure that the cache lookup based on the hash only is a good idea. If this is done so many times that computing the hash becomes the bottleneck, maybe it is better to simply cache the has.

except KeyError:
inst = cls(name, subproject, machine, _hash)
_OPTIONS_CACHE[_hash] = inst
return inst

@classmethod
@lru_cache(None)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this has a big impact. It is probably only used for parsing options given by the command line.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This gets used in a surprising number of places, many of them because we still are doing user input validation below the interpreter.

def from_string(cls, raw: str) -> 'OptionKey':
"""Parse the raw command line format into a three part tuple.

Expand Down