Module: Assistant::CommentManifest
- Defined in:
- app/services/assistant/comment_manifest.rb
Overview
YAML-backed source of truth for database object/column comments and
first-pass AI safety metadata (e.g. restricted columns).
Constant Summary collapse
- MANIFEST_DIR =
Manifest dir.
Rails.root.join('db/comments').freeze
- SUPPORTED_EXTENSIONS =
Supported extensions.
%w[.yml .yaml].freeze
Class Method Summary collapse
-
.apply(object_name: nil, dry_run: false, connection: ActiveRecord::Base.lease_connection) ⇒ Array<String>
Generate and optionally execute COMMENT ON SQL statements.
-
.build_domain_map ⇒ Hash{String => Set<String>}
Scan all manifests and build a domain to object names map.
-
.build_relkind_map(connection) ⇒ Hash{String => String}
Query pg_class for the relkind of every public relation.
-
.column_comments(relation:, schema_name:, connection:) ⇒ Hash{String => String}
Fetch current comments for all columns of a relation.
-
.column_types(relation, schema_name: 'public', connection: ActiveRecord::Base.lease_connection) ⇒ Hash{String => String}
Read column names and SQL types from pg_attribute + pg_type.
-
.details(object_name, connection: ActiveRecord::Base.lease_connection) ⇒ Hash?
Detailed schema for a specific object.
-
.diff(object_name: nil, connection: ActiveRecord::Base.lease_connection) ⇒ Array<Hash>
Compare manifest comments against the live database comments.
-
.domain_for(object_name) ⇒ Array<String>
Domains declared for an object.
-
.domain_map ⇒ Hash{String => Set<String>}
Cached mapping of domain name to object names.
-
.dump_model_tables(output_dir: MANIFEST_DIR, connection: ActiveRecord::Base.lease_connection, object_name: nil) ⇒ Array<String>
Generate YAML manifests for all (or one) ApplicationRecord table(s).
-
.load_manifest(object_name) ⇒ Hash?
Load and parse the YAML manifest for an object.
-
.manifest_exists?(object_name) ⇒ Boolean
Check whether a YAML manifest exists for the given object.
-
.manifest_path(object_name) ⇒ Pathname?
Find the filesystem path for an object's manifest.
-
.normalize_column_config(config) ⇒ Hash{String => Object}
Normalize a column configuration to a string-keyed hash.
-
.object_names ⇒ Array<String>
All manifest object names (basename of each YAML file).
-
.objects_for_domain(domain_name) ⇒ Set<String>
Objects that declare the given domain.
-
.objects_for_domains(domain_names) ⇒ Set<String>
Objects that declare any of the given domains.
-
.qualified_relation_identifier(schema_name:, relation:, connection:) ⇒ String
Build a fully qualified, quoted relation identifier.
-
.relation_comment(relation:, schema_name:, connection:) ⇒ String?
Fetch the current comment on a relation.
-
.relation_keyword_for(relkind) ⇒ String
Convert a pg_class.relkind value to a COMMENT ON keyword.
-
.relation_kind(relation:, schema_name:, connection:) ⇒ String?
Look up the pg_class.relkind for a relation.
-
.relation_type_label(relkind) ⇒ String
Convert pg_class.relkind to a human-readable type label.
-
.relkind_map(connection: ActiveRecord::Base.lease_connection) ⇒ Hash{String => String}
Batch-load relkind for all public relations in one query.
-
.reset_domain_map! ⇒ nil
Clear the cached domain map.
-
.reset_schema_cache! ⇒ nil
Clear cached schema introspection data.
-
.resolve_targets(object_name) ⇒ Array<String>
Normalize an optional object name into a list of manifest targets.
-
.restricted_columns_for_objects(object_names:) ⇒ Array<String>
Columns marked as restricted across the requested objects.
-
.sql_statements_for(object_name:, connection: ActiveRecord::Base.lease_connection) ⇒ Array<String>
Build COMMENT ON SQL statements for a manifest.
-
.summary(connection: ActiveRecord::Base.lease_connection) ⇒ Array<Hash>
Compact summary of all manifest-backed objects.
-
.sync(object_name: nil, connection: ActiveRecord::Base.lease_connection) ⇒ Hash
Synchronize YAML manifests with actual database columns.
-
.to_sql_comment(value, connection) ⇒ String
Convert a Ruby value to a SQL comment literal (or NULL).
Class Method Details
.apply(object_name: nil, dry_run: false, connection: ActiveRecord::Base.lease_connection) ⇒ Array<String>
Generate and optionally execute COMMENT ON SQL statements.
162 163 164 165 166 167 168 169 170 |
# File 'app/services/assistant/comment_manifest.rb', line 162 def apply(object_name: nil, dry_run: false, connection: ActiveRecord::Base.lease_connection) # :reek:BooleanParameter targets = resolve_targets(object_name) statements = targets.flat_map { |target| sql_statements_for(object_name: target, connection: connection) } return statements if dry_run statements.each { |sql| connection.execute(sql) } statements end |
.build_domain_map ⇒ Hash{String => Set<String>}
Scan all manifests and build a domain to object names map.
576 577 578 579 580 581 582 583 584 585 |
# File 'app/services/assistant/comment_manifest.rb', line 576 def build_domain_map map = Hash.new { |by_domain, k| by_domain[k] = Set.new } object_names.each do |name| domains = domain_for(name) domains.each { |domain| map[domain] << name } end map end |
.build_relkind_map(connection) ⇒ Hash{String => String}
Query pg_class for the relkind of every public relation.
544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 |
# File 'app/services/assistant/comment_manifest.rb', line 544 def build_relkind_map(connection) sql = <<~SQL.squish SELECT c.relname, c.relkind FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace WHERE n.nspname = 'public' SQL connection.select_all(sql).to_a.to_h do |row| [row['relname'], row['relkind']] end rescue StandardError => e Rails.logger.warn("[Assistant::CommentManifest] Failed to load relkind map: #{e.}") {} end |
.column_comments(relation:, schema_name:, connection:) ⇒ Hash{String => String}
Fetch current comments for all columns of a relation.
343 344 345 346 347 348 349 350 351 352 353 354 355 356 |
# File 'app/services/assistant/comment_manifest.rb', line 343 def column_comments(relation:, schema_name:, connection:) sql = <<~SQL.squish SELECT a.attname AS column_name, d.description AS comment FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace JOIN pg_attribute a ON a.attrelid = c.oid AND a.attnum > 0 AND NOT a.attisdropped LEFT JOIN pg_description d ON d.objoid = c.oid AND d.objsubid = a.attnum WHERE n.nspname = #{connection.quote(schema_name)} AND c.relname = #{connection.quote(relation)} ORDER BY a.attnum SQL connection.select_rows(sql).to_h end |
.column_types(relation, schema_name: 'public', connection: ActiveRecord::Base.lease_connection) ⇒ Hash{String => String}
Read column names and SQL types from pg_attribute + pg_type.
500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 |
# File 'app/services/assistant/comment_manifest.rb', line 500 def column_types(relation, schema_name: 'public', connection: ActiveRecord::Base.lease_connection) sql = <<~SQL.squish SELECT a.attname AS column_name, pg_catalog.format_type(a.atttypid, a.atttypmod) AS data_type FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace JOIN pg_attribute a ON a.attrelid = c.oid AND a.attnum > 0 AND NOT a.attisdropped WHERE n.nspname = #{connection.quote(schema_name)} AND c.relname = #{connection.quote(relation)} ORDER BY a.attnum SQL connection.select_all(sql).to_a.to_h do |row| [row['column_name'], row['data_type']] end end |
.details(object_name, connection: ActiveRecord::Base.lease_connection) ⇒ Hash?
Detailed schema for a specific object.
Merges pg_attribute types with YAML manifest comments.
467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 |
# File 'app/services/assistant/comment_manifest.rb', line 467 def details(object_name, connection: ActiveRecord::Base.lease_connection) manifest = load_manifest(object_name) return nil unless manifest schema_name = manifest.fetch('schema', 'public') relation = manifest.fetch('object_name', object_name) relkind = relation_kind(relation: relation, schema_name: schema_name, connection: connection) return nil unless relkind types = column_types(relation, schema_name: schema_name, connection: connection) manifest_columns = manifest.fetch('columns', {}) columns = types.each_with_object({}) do |(col_name, col_type), out| manifest_cfg = manifest_columns[col_name] comment = manifest_cfg.is_a?(Hash) ? manifest_cfg['comment'] : manifest_cfg out[col_name] = comment.present? ? "#{col_type} — #{comment}" : col_type end { name: object_name, description: manifest['comment'], columns: columns, tips: manifest['tips'], type: relation_type_label(relkind) }.compact end |
.diff(object_name: nil, connection: ActiveRecord::Base.lease_connection) ⇒ Array<Hash>
Compare manifest comments against the live database comments.
177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 |
# File 'app/services/assistant/comment_manifest.rb', line 177 def diff(object_name: nil, connection: ActiveRecord::Base.lease_connection) resolve_targets(object_name).flat_map do |target| manifest = load_manifest(target) next [] unless manifest schema_name = manifest.fetch('schema', 'public') relation = manifest.fetch('object_name', target) relkind = relation_kind(relation: relation, schema_name: schema_name, connection: connection) next [{ object: target, issue: 'missing_relation' }] if relkind.nil? expected_relation_comment = manifest['comment'] actual_relation_comment = relation_comment(relation: relation, schema_name: schema_name, connection: connection) diffs = [] if expected_relation_comment != actual_relation_comment diffs << { object: target, type: 'relation_comment', expected: expected_relation_comment, actual: actual_relation_comment } end expected_columns = manifest.fetch('columns', {}) actual_columns = column_comments(relation: relation, schema_name: schema_name, connection: connection) expected_columns.each do |column_name, config| expected_comment = normalize_column_config(config)['comment'] actual_comment = actual_columns[column_name.to_s] next if expected_comment == actual_comment diffs << { object: target, type: 'column_comment', column: column_name.to_s, expected: expected_comment, actual: actual_comment } end diffs end end |
.domain_for(object_name) ⇒ Array<String>
Domains declared for an object.
59 60 61 62 63 64 65 66 67 |
# File 'app/services/assistant/comment_manifest.rb', line 59 def domain_for(object_name) manifest = load_manifest(object_name) return [] unless manifest raw = manifest['domain'] return [] if raw.nil? Array(raw).map(&:to_s) end |
.domain_map ⇒ Hash{String => Set<String>}
Cached mapping of domain name to object names.
91 92 93 94 95 96 97 |
# File 'app/services/assistant/comment_manifest.rb', line 91 def domain_map if Rails.env.local? build_domain_map else @domain_map ||= build_domain_map end end |
.dump_model_tables(output_dir: MANIFEST_DIR, connection: ActiveRecord::Base.lease_connection, object_name: nil) ⇒ Array<String>
Generate YAML manifests for all (or one) ApplicationRecord table(s).
226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 |
# File 'app/services/assistant/comment_manifest.rb', line 226 def dump_model_tables(output_dir: MANIFEST_DIR, connection: ActiveRecord::Base.lease_connection, object_name: nil) Rails.application.eager_load! table_names = if object_name.present? [object_name.to_s] else ApplicationRecord.descendants .reject(&:abstract_class?) .filter_map(&:table_name) .map(&:to_s) .uniq .sort end FileUtils.mkdir_p(output_dir) table_names.each do |table_name| next unless relation_kind(relation: table_name, schema_name: 'public', connection: connection) payload = { 'schema' => 'public', 'object_name' => table_name, 'comment' => relation_comment(relation: table_name, schema_name: 'public', connection: connection), 'columns' => column_comments(relation: table_name, schema_name: 'public', connection: connection).to_h do |column_name, comment| [column_name, { 'comment' => comment, 'restricted' => false }] end } File.write(output_dir.join("#{table_name}.yml"), payload.to_yaml) end end |
.load_manifest(object_name) ⇒ Hash?
Load and parse the YAML manifest for an object.
291 292 293 294 295 296 |
# File 'app/services/assistant/comment_manifest.rb', line 291 def load_manifest(object_name) path = manifest_path(object_name) return nil unless path YAML.safe_load_file(path.to_s) || {} end |
.manifest_exists?(object_name) ⇒ Boolean
Check whether a YAML manifest exists for the given object.
22 23 24 |
# File 'app/services/assistant/comment_manifest.rb', line 22 def manifest_exists?(object_name) manifest_path(object_name).present? end |
.manifest_path(object_name) ⇒ Pathname?
Find the filesystem path for an object's manifest.
423 424 425 426 427 428 429 430 |
# File 'app/services/assistant/comment_manifest.rb', line 423 def manifest_path(object_name) stem = object_name.to_s.downcase SUPPORTED_EXTENSIONS.each do |ext| candidate = MANIFEST_DIR.join("#{stem}#{ext}") return candidate if File.exist?(candidate) end nil end |
.normalize_column_config(config) ⇒ Hash{String => Object}
Normalize a column configuration to a string-keyed hash.
389 390 391 392 393 394 395 396 |
# File 'app/services/assistant/comment_manifest.rb', line 389 def normalize_column_config(config) case config when Hash config.transform_keys(&:to_s) else { 'comment' => config, 'restricted' => false } end end |
.object_names ⇒ Array<String>
All manifest object names (basename of each YAML file).
29 30 31 32 33 |
# File 'app/services/assistant/comment_manifest.rb', line 29 def object_names Dir.glob(MANIFEST_DIR.join('*.y{a,}ml').to_s).map do |path| File.basename(path, File.extname(path)).downcase end.sort end |
.objects_for_domain(domain_name) ⇒ Set<String>
Objects that declare the given domain.
73 74 75 |
# File 'app/services/assistant/comment_manifest.rb', line 73 def objects_for_domain(domain_name) domain_map[domain_name.to_s] || Set.new end |
.objects_for_domains(domain_names) ⇒ Set<String>
Objects that declare any of the given domains.
81 82 83 84 85 86 |
# File 'app/services/assistant/comment_manifest.rb', line 81 def objects_for_domains(domain_names) map = domain_map Array(domain_names).each_with_object(Set.new) do |name, set| set.merge(map[name.to_s] || []) end end |
.qualified_relation_identifier(schema_name:, relation:, connection:) ⇒ String
Build a fully qualified, quoted relation identifier.
415 416 417 |
# File 'app/services/assistant/comment_manifest.rb', line 415 def qualified_relation_identifier(schema_name:, relation:, connection:) "#{connection.quote_table_name(schema_name)}.#{connection.quote_table_name(relation)}" end |
.relation_comment(relation:, schema_name:, connection:) ⇒ String?
Fetch the current comment on a relation.
323 324 325 326 327 328 329 330 331 332 333 334 335 |
# File 'app/services/assistant/comment_manifest.rb', line 323 def relation_comment(relation:, schema_name:, connection:) sql = <<~SQL.squish SELECT d.description FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace LEFT JOIN pg_description d ON d.objoid = c.oid AND d.objsubid = 0 WHERE n.nspname = #{connection.quote(schema_name)} AND c.relname = #{connection.quote(relation)} LIMIT 1 SQL connection.select_value(sql) end |
.relation_keyword_for(relkind) ⇒ String
Convert a pg_class.relkind value to a COMMENT ON keyword.
376 377 378 379 380 381 382 383 |
# File 'app/services/assistant/comment_manifest.rb', line 376 def relation_keyword_for(relkind) case relkind when 'm' then 'MATERIALIZED VIEW' when 'v' then 'VIEW' when 'f' then 'FOREIGN TABLE' else 'TABLE' end end |
.relation_kind(relation:, schema_name:, connection:) ⇒ String?
Look up the pg_class.relkind for a relation.
304 305 306 307 308 309 310 311 312 313 314 315 |
# File 'app/services/assistant/comment_manifest.rb', line 304 def relation_kind(relation:, schema_name:, connection:) sql = <<~SQL.squish SELECT c.relkind FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace WHERE n.nspname = #{connection.quote(schema_name)} AND c.relname = #{connection.quote(relation)} LIMIT 1 SQL connection.select_value(sql) end |
.relation_type_label(relkind) ⇒ String
Convert pg_class.relkind to a human-readable type label.
564 565 566 567 568 569 570 571 |
# File 'app/services/assistant/comment_manifest.rb', line 564 def relation_type_label(relkind) case relkind when 'm' then 'materialized_view' when 'v' then 'view' when 'f' then 'foreign_table' else 'table' end end |
.relkind_map(connection: ActiveRecord::Base.lease_connection) ⇒ Hash{String => String}
Batch-load relkind for all public relations in one query.
Cached in production, fresh in dev/test.
532 533 534 535 536 537 538 |
# File 'app/services/assistant/comment_manifest.rb', line 532 def relkind_map(connection: ActiveRecord::Base.lease_connection) if Rails.env.local? build_relkind_map(connection) else @relkind_map ||= build_relkind_map(connection) end end |
.reset_domain_map! ⇒ nil
Clear the cached domain map.
102 103 104 |
# File 'app/services/assistant/comment_manifest.rb', line 102 def reset_domain_map! @domain_map = nil end |
.reset_schema_cache! ⇒ nil
Clear cached schema introspection data.
520 521 522 |
# File 'app/services/assistant/comment_manifest.rb', line 520 def reset_schema_cache! @relkind_map = nil end |
.resolve_targets(object_name) ⇒ Array<String>
Normalize an optional object name into a list of manifest targets.
363 364 365 366 367 368 369 370 |
# File 'app/services/assistant/comment_manifest.rb', line 363 def resolve_targets(object_name) return object_names if object_name.blank? target = object_name.to_s.downcase raise ArgumentError, "No manifest found for #{target}" unless manifest_exists?(target) [target] end |
.restricted_columns_for_objects(object_names:) ⇒ Array<String>
Columns marked as restricted across the requested objects.
39 40 41 42 43 44 45 46 47 48 49 |
# File 'app/services/assistant/comment_manifest.rb', line 39 def restricted_columns_for_objects(object_names:) Array(object_names).flat_map do |object_name| manifest = load_manifest(object_name) next [] unless manifest manifest.fetch('columns', {}).filter_map do |column_name, config| cfg = normalize_column_config(config) column_name.to_s.downcase if cfg['restricted'] == true end end.uniq end |
.sql_statements_for(object_name:, connection: ActiveRecord::Base.lease_connection) ⇒ Array<String>
Build COMMENT ON SQL statements for a manifest.
263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 |
# File 'app/services/assistant/comment_manifest.rb', line 263 def sql_statements_for(object_name:, connection: ActiveRecord::Base.lease_connection) manifest = load_manifest(object_name) return [] unless manifest schema_name = manifest.fetch('schema', 'public') relation = manifest.fetch('object_name', object_name) relkind = relation_kind(relation: relation, schema_name: schema_name, connection: connection) return [] if relkind.nil? relation_identifier = qualified_relation_identifier(schema_name: schema_name, relation: relation, connection: connection) relation_keyword = relation_keyword_for(relkind) statements = [] statements << "COMMENT ON #{relation_keyword} #{relation_identifier} IS #{to_sql_comment(manifest['comment'], connection)};" manifest.fetch('columns', {}).each do |column_name, config| comment = normalize_column_config(config)['comment'] column_identifier = connection.quote_column_name(column_name.to_s) statements << "COMMENT ON COLUMN #{relation_identifier}.#{column_identifier} IS #{to_sql_comment(comment, connection)};" end statements end |
.summary(connection: ActiveRecord::Base.lease_connection) ⇒ Array<Hash>
Compact summary of all manifest-backed objects.
440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 |
# File 'app/services/assistant/comment_manifest.rb', line 440 def summary(connection: ActiveRecord::Base.lease_connection) relkinds = relkind_map(connection: connection) object_names.filter_map do |name| manifest = load_manifest(name) next unless manifest relkind = relkinds[manifest.fetch('object_name', name)] next unless relkind # object doesn't exist in the database { name: name, description: manifest['comment'], column_count: manifest.fetch('columns', {}).size, type: relation_type_label(relkind), domain: Array(manifest['domain']) } end end |
.sync(object_name: nil, connection: ActiveRecord::Base.lease_connection) ⇒ Hash
Synchronize YAML manifests with actual database columns.
Adds new columns (with comment: nil), removes columns no longer in the DB,
and re-orders YAML columns to match the database column order.
114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 |
# File 'app/services/assistant/comment_manifest.rb', line 114 def sync(object_name: nil, connection: ActiveRecord::Base.lease_connection) targets = object_name ? [object_name.to_s.downcase] : object_names changes = { added: {}, removed: {}, skipped: [] } targets.each do |name| manifest = load_manifest(name) next unless manifest schema_name = manifest.fetch('schema', 'public') relation = manifest.fetch('object_name', name) relkind = relation_kind(relation: relation, schema_name: schema_name, connection: connection) unless relkind changes[:skipped] << name next end db_columns = column_types(relation, schema_name: schema_name, connection: connection) manifest_columns = manifest.fetch('columns', {}) new_cols = db_columns.keys - manifest_columns.keys removed_cols = manifest_columns.keys - db_columns.keys next if new_cols.empty? && removed_cols.empty? # Rebuild columns hash in DB column order, preserving existing config ordered_columns = {} db_columns.each_key do |col| ordered_columns[col] = manifest_columns.fetch(col, { 'comment' => nil }) end manifest['columns'] = ordered_columns path = manifest_path(name) File.write(path, manifest.to_yaml) changes[:added][name] = new_cols unless new_cols.empty? changes[:removed][name] = removed_cols unless removed_cols.empty? end changes end |
.to_sql_comment(value, connection) ⇒ String
Convert a Ruby value to a SQL comment literal (or NULL).
403 404 405 406 407 |
# File 'app/services/assistant/comment_manifest.rb', line 403 def to_sql_comment(value, connection) return 'NULL' if value.nil? connection.quote(value.to_s) end |